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PREFACE 


This  paper  is  an  introd  uction  and  general  overview  of  the  Training  Impact  Decision 
System  (TIDES)  as  a  decision  support  system  (DSS)  for  use  by  Air  Force  decision 
makers  involved  with  critical  manpower,  personnel,  and  training  decisions.  This  report 
describes  work  currently  underway  under  contracts  F33615-89-C-0001  and  F41624-93- 
C-5009  and  briefly  highlights  previous  activities  conducted  as  part  of  earlier  exploratory 
research  and  development  efforts  such  as  the  Training  Decisions  System  and  Training 
Decisions  Modeling  Technologies.  For  more  detailed  technical  information,  see  the 
publications  referenced  in  this  document,  the  TIDES  specifications  documents,  or  the 
references  noted  in  Appendix  A. 

A  shorter  version  of  this  paper  (Grimes,  Wimpee,  Bennett,  &  Mitchell,  1994)  was 
presented  by  Lt  Col  Wimpee  at  the  1994  Symposium  on  Human  Interaction  on  Complex 
Systems,  North  Carolina  A&T  State  University,  Greensboro,  NC,  in  September  1994. 

Most  of  the  conceptual  design,  mathematical  modeling,  and  initial  software 
development  involved  in  this  series  of  Training  Decisions  projects,  was  the  work  of  the 
late  Dr.  DAVID  S.  VAUGHAN  (25  November  1949  -  15  January  1994),  McDonnell 
Douglas  Corporation.  It  is  with  sincere  gratitude  for  his  outstanding  contributions  that 
this  work  continues. 


SUMMARY 


This  report  documents  continuing  research  and  applfcation  of  a  decision  support 
system  (DSS)  by  the  Armstrong  Laboratory,  Human  Resources  Directorate,  under 
contracts  F33615-89-C-0001  and  F41 624-93 -C-5009.  This  DSS  is  the  Training 
Impact  Decision  System  (TIDES),  a  computer-based  system  specifically  designed 
to  support  key  planners  and  decision  makers  (functional  managers  at  the  Air  Staff 
or  Major  Command,  training  managers,  etc.)  for  a  career  specialty  (occupation). 
The  TIDES  was  developed  to  extend  task-based  Job  analysis  to  model  utilization 
and  training  (U&T)  programs  and  quantify  specialty  training  requirements, 
including  on-the-job  training  (OJT).  This  type  of  career  field  modeling  facilitates 
more  systematic  consideration  of  personnel  utilization  factors,  training  costs, 
resource  requirements,  training  capacities  of  units  (base-level  organizations)  and 
managers  preferences  in  U«&T  Workshops  where  career  field  decision  makers 
determine  the  optimal  allocation  of  personnel  and  training  resources. 
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INTRODUCTION 


Human  Resources  Management  (HRM)  and  other  organizational  decisions  are  often  made  for 
a  desired  effect  on  the  profit  and  loss  statement  without  regard  to  their  potential  impacts  on  other 
HRM  programs  such  as  selection,  placement,  training,  or  occupational  structures.  Such  HRM 
decisions  can  be  modeled  in  terms  of  possible  changes  in  job  content  and  changes  in  training 
requirements.  Such  models  can  be  quantified  in  order  to  predict  the  impact  of  the  changes  on 
both  formal  and  on-the-job  training  (OJT)  programs;  ultimately  these  impacts  must  be  assessed  in 
terms  of  costs  and  organizational  training  capacities.  This  type  of  occupational  modeling  also 
permits  estimation  of  the  costs  of  training  on-the-job  which  previously  have  not  been  visible  to 
most  managers  or  trainers.  By  making  such  hidden  costs  visible  and  by  quantifying  the  impact  of 
various  proposed  changes,  such  modeling  efforts  can  support  managers  in  making  more  realistic 
and  cost-efifective  decisions  (Rueter,  Feldsott,  &  Vaughan,  1989;  Mitchell,  Yadrick,  &  Bennett, 
1993). 

Training  management  and  planning  have  to  be  an  integral  part  of  any  large  organization.  The 
very  sizable,  largely  hidden  costs  of  OJT  make  management  of  such  training  very  critical.  This  is 
a  problem  of  some  magnitude  which  has  only  recently  been  recognized  in  the  civilian  literature. 
The  American  Society  for  Training  &  Development  (ASTD)  and  the  U.S.  Department  of  Labor 
(DOL)  estimate  that  employers  in  the  U.S.  "spend  $30  billion  a  year  on  formal  training  and 
approximately  $180  billion  on  informal  OJT  (ASTD,  1988)."  Clearly,  any  improvement  in  the 
management  of  training  (including  OJT)  has  the  potential  for  significant  savings  and  substantially 
enhanced  productivity  (Campbell,  Dunnette,  Lawler,  &  Weick,  1970). 

At  the  same  time,  recent  developments  in  occupational  analysis  and  training  research,  as  well 
as  in  decision  making  processes,  have  created  new  opportunities  for  optimizing  job  training 
(Mitchell,  Ruck,  &  Driskill,  1988).  Such  developments  include  the  recent  emergence  of  the 
Instructional  System  Development  (ISD)  process  as  a  systematic  approach  to  defining  educational 
and  training  objectives  for  making  and  coordinating  major  training  decisions  (Eschenbrenner, 
DeVries,  Miller  &  Ruck,  1980;  Vaughan,  Mitchell,  Yadrick,  Perrin,  Knight,  Eschenbrenner, 
Rueter,  &  Feldsott,  1989).  Such  innovative  procedural  changes  also  make  obvious  a  need  for 
technologically-advanced  data  generation,  analysis,  and  evaluation  to  support  military  decision 
makers. 

BACKGROUND 
Decision  Support  Systems 

Recent  technological  developments  in  personal  computers,  computer  networks,  and  computer- 
based  models  have  made  possible  a  level  of  interactive  support  for  decision  makers  never  before 
possible  (Carlson,  1983).  Decision  support  for  management  decision  making  implies  the  use  of 
computers  to:  1)  assist  managers  in  their  decision  processes  in  semi-structured  tasks;  2)  support, 
rather  than  replace,  managerial  judgment;  and  3)  improve  the  effectiveness  of  decision  making 
rather  than  its  efficiency  (Keen  and  Scott  Morton,  1978).  Additionally,  Sprague  and  Carlson 
(1982)  observed  that  decision  support  systems  (DSSs)  may  be  defined  by  their  capabilities  in 
several  critical  areas,  including  the  following:  1)  they  tend  to  be  aimed  at  the  less  well  structured, 
under-specified  problems  that  upper-level  managers  typically  face;  2)  they  attempt  to  combine  the 
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use  of  models  or  analytic  techniques  with.traditional  data  access  and  retrieval  functions;  3)  they  ■ 
specifically  focus  on  features  that  make  them  easy  to  use  by  non-computer  people  in  an  interactive 
mode;  and  4)  they  emphasize  flexibility  and  adaptability  to  accommodate  changes  in  the 
environment  and  decision-making  approach  of  the  user. 

To  be  usefiil  to  Air  Force  decision  makers,  a  DSS  should  make  maximum  use  of  existing  data 
sources,  should  permit  collection  or  estimation  of  new  data  when  needed,  and  should  organize  or 
structure  the  information  in  ways  which  are  meaningfol  to  those  involved.  Fortunately,  the  Air 
Force  has  a  number  of  on-going  programs  which  generate  information  which  can  be  and  are  used 
in  making  manpower,  personnel,  and  training  decisions. 

Air  Force  Job  Analysis 

Over  the  last  three  decades,  the  Air  Force  has  developed  a  task-based  approach  for  studying 
Air  Force  occupations  (Christal,  1974;  Morsh,  1964).  As  part  of  this  occupational  analysis  (OA) 
process,  tasks  are  defined  by  job  experts  in  their  own  technical  terminology.  Several  kinds  of  data 
are  collected  from  job  incumbents  and  supervisors  on  these  tasks.  For  example  incumbents 
provide  information  about  tasks  they  perform  in  their  present  jobs  and  the  relative  amount  of  their 
job  time  spent  performing  such  tasks.  Senior  technicians  rate  specialty  tasks  as  to  their  relative 
difficulty  to  learn  and  needed  emphasis  in  initial  skills  training.  Resulting  data  can  be  used  to 
examine  the  variety  of  specialized  jobs  within  an  occupation,  to  assess  how  jobs  change  at 
advanced  skill  levels,  and  to  review  official  occupational  descriptions  and  initial  skills  training 
programs  (Christal  &  Weissmuller,  1988;  Mitchell,  et  al.,  1988). 

Air  Force  Training 

The  Air  Force  currently  has  an  extensive,  very  highly  developed  training  system  which 
prepares  individuals  to  perform  a  wide  variety  of  jobs;  most  of  the  Air  Force  training  programs 
are  organized  by  specialty  (occupation).  There  are  four  primary  settings  for  most  Air  Force 
training;  the  first  is  formal  classroom  instruction.  A  second  major  category  is  supervised  on-the- 
job  training  (OJT).  Third,  there  are  "hands-on"  training  programs  (laboratory,  mobile  training 
teams,  field  exercises,  etc.)  where  equipment-  or  system-specific  training  is  conducted,  quite  often 
at  or  near  the  work  site.  The  final  type  of  training  involves  self-study  (correspondence,  self-paced 
computer  lessons,  etc)  courses.  [A  number  of  other  training  delivery  techniques  might  also  be 
considered  for  specific  occupations  (distance  learning,  etc.);  these  can  be  grouped  into  the  four 
broad  categories  listed  above  or  considered  as  separate  training  settings.] 

To  make  good  decisions  about  the  training  needed  for  an  occupation  or  system,  decision 
makers  must  develop  a  framework  or  "model"  as  a  means  of  visualizing  and  understanding  the 
jobs  and  training  programs  of  the  occupation  or  system  under  consideration.  The  framework  or 
"model"  also  includes  the  occupation's  technical  training  and  general  professional  development 
requirements,  as  well  as  the  relative  costs  and  payoffs  of  various  training  configurations.  Such  a 
"model"  provides  a  concise  summary  of  the  current  status  of  the  occupation,  creates  a  common 
"language"  for  discussion  or  negotiation,  and  forms  the  baseline  against  which  various  alternative 
proposals  for  refinements  to  training  programs,  changes  of  the  job  structure,  or  reallocation  of 
personnel  and  resources,  can  be  evaluated  (Ruck  &  Birdlebough,  1 977 ;  Chin,  Lamb,  Bennett,  & 
Vaughan,  1992).  Therefore,  a  decision  support  system,  which  incorporates  a  modeling  capability. 
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is  needed  to  provide  information  to  key  decision  makers  to  help  them  assess  potential  outcomes  - 
of  the  various  alternative  proposals  (Mitchell,  Vaughan,  Knight,  Rueter,  Fast,  Haynes,  &  Bennett, 
1992). 

TRAINING  IMPACT  DECISION  SYSTEM  (TIDES) 

Technology  Description 

The  TIDES  technology  helps  Air  Force  training  managers  balance  an  occupation's  training 
needs  versus  resources  and  requirements  to  optimize  training  management  throughout  the  entire 
career  of  an  individual  or  group  of  individuals  (this  includes  all  training,  formal  and  informal,  as 
well  as  all  jobs  an  individual  could  perform  throughout  their  career  with  the  Air  Force).  Such 
specialized  groups  of  individuals  have  different  training  needs,  and  substantial  economies  are 
possible  through  tailored  training,  TIDES  technology  provides  analysts  and  decision  makers  with 
a  tool  to  systematically  gather,  integrate,  and  analyze  information  about  jobs,  tasks,  career 
assignments,  personnel  flows,  and  technical  training  programs  within  an  Air  Force  occupation  (see 
Figure  1). 


Figure  1.  TIDES  Operational  Concept  -  Modeling  Current  &  Alternative  Utilization  and 
Training  Patterns. 
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By  dynamically  modeling  an  occupation's  career  flow  patterns,  the  TIDES  technology 
provides  a  "what  if  capability  to  assess  the  long-term  impact  of  current  and  future  constraints 
stemming  from  changing  training,  personnel  and  fiscal  policies  and  resources.  TIDES  analyses 
also  aid  decision  makers  in  determining  what  tasks  associated  with  an  occupation  to  train,  when 
to  provide  that  training  (at  what  career  points),  and  the  method  best  suited  for  that  type  of 
training  (Ruck,  1982;  Vaughan,  Yadrick,  Perrin,  Cooley,  Dunteman,  Clark,  &  Rueter,  1984). 

TBOES  Data  Collection 

As  part  of  the  TIDES  data  collection  process,  analysts  identify  the  resource  requirements 
needed  to  conduct  the  necessary  type  of  training.  To  collect  this  information,  groups  of  subject 
matter  experts  (SMEs)  within  an  Air  Force  occupational  area  are  assembled.  These  SMEs  first 
identify  the  jobs  and  training  courses  for  a  particular  Air  Force  occupation.  Next,  they  describe 
the  jobs  and  training  courses  which  are  identified  for  the  occupation. 

The  TIDES  technology  uses  task  modules  (TMs),  groups  of  tasks  that  share  similar 
characteristics  (e.g.,  equipment,  skills,  etc.).  These  tasks  are  collected  as  part  of  the  Air  Force 
occupational  process.  The  current  training  course  and  job  assignment  flows  must  be  identified  in 
order  to  build  a  model  of  the  utilization  and  training  (U&T)  pattern  for  the  occupation.  SMEs 
also  validate  the  components  of  the  model-the  positions  within  the  occupation  Oobs),  skills  and 
experience  required  to  perform  those  jobs,  and  number  of  years  a  person  is  likely  to  hold  that  job. 
Finally,  the  SMEs  identify  locations  that  are  considered  to  be  "typical"  or  representative  sites  to 
assess  typical  OJT  programs.  This  information  is  used  as  a  baseline  for  future  comparisons.  This 
allows  TIDES  to  describe  each  of  the  training  courses  and  training  patterns,  within  an  occupation, 
in  terms  of  cost  and  resource  requirements. 

Cost  estimates  for  conducting  training  are  obtained  by  collecting  information  on  travel  to  and 
from  the  training  sites  and  the  per  diem  cost  while  students  attend  the  training.  Information  is 
also  obtained  on  the  variable  cost  per  student  per  formal  training  site,  as  well  as  the  number  of 
hours  and  wage  rates  of  individuals  (trainers  &  trainees)  involved  in  on-the-job  training.  All  cost 
information  is  gathered  for  each  TM  per  each  training  setting.  In  addition  to  the  detailed 
information  related  to  cost,  information  is  also  collected  on  the  number  of  hours  required  to  train 
individuals  to  certain  levels  of  proficiency  (learning  curves)  by  TM  and  training  setting  (Perrin, 
Knight,  Mitchell,  Vaughan,  &  Yadrick,  1988).  Using  this  information,  tradeoffs  between  different 
training  settings  with  respect  to  a  given  level  of  desired  performance  can  be  identified.  For 
example,  if  a  portion  of  training  were  moved  from  one  setting  to  another,  the  TIDES  can  estimate 
the  approximate  amount  of  time  and  expense  involved  in  each  setting  by  desired  proficiency  level. 

All  of  these  types  of  information  are  synthesized  into  a  set  of  TIDES  files  which  quantitatively 
model  the  current  Utilization  and  Training  (U&T)  pattern  of  a  career  field  (left  half  of  Figure  1). 
The  U&T  simulation  is  then  run  and  reports  generated  on  the  total  annual  training  costs  for  the 
AFS  and  training  capacity  (including  identifying  critical  constraints).  Such  summary  data  and 
reports  become  the  "baseline"  against  which  any  proposed  changes  to  specialty  jobs  and  training 
programs  can  be  assessed. 
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TIDES  "What  If  Modeling 


In  general,  TIDES  "what  if  modeling  involves  creating  alternative  job  and  training  patterns 
reflecting  manpower,  personnel,  and  training  policy  options  of  interest.  This  can  be  done  by  a 
TIDES  analyst  or  modeler,  or  by  Air  Force  decision  makers,  such  as  the  functional  manager  for 
the  AFS  at  the  Air  Staff  or  Major  Command,  or  in  a  group  session  such  at  a  U&T  Workshop 
(right  side  of  Figure  1).  Once  an  alternative  is  specified,  the  model  is  run  on  the  alternative  job 
and  training  patterns,  and  model  results  are  compared  to  those  for  the  baseline  model  (the  current 
job  and  training  pattern).  In  creating  an  alternative  job  and  training  pattern,  new  values  are 
specified  for  certain  input  or  independent  variables  in  the  model  (Vaughan,  et  al.,  1989). 

While  such  relationships  may  have  been  intuitively  obvious  to  some  managers  in  the  past,  it  is 
only  now  that  such  relationships  can  be  quantified  and  reported  to  support  and  justify  necessary 
training  investment  decisions.  The  types  of  information  and  data  displays  provided  in  a  decision 
support  system  such  as  the  TIDES  can  be  crucial  for  helping  managers  and  other  decision  makers 
improve  their  understanding  of  the  potential  impact  or  consequences  of  their  decisions.  Further, 
such  a  system  will  also  help  them  document  the  rationale  for  such  decisions,  and  defend  such 
decisions  to  higher  levels  of  command  within  the  Department  of  Defense  and  to  Congress. 

TIDES  Success 

During  the  development  of  TIDES,  tests  were  run  on  several  Air  Force  occupations,  ranging 
from  fairly  homogeneous  career  fields,  such  as  the  B-1  Avionics  Test  Station  (two  jobs  plus 
supervisors),  to  very  structured  specialties,  such  as  Security  and  Law  Enforcement  (85  job 
variations),  to  complex  electronics  career  fields,  such  as  the  Radar  and  Inertial  Navigation 
Systems  technicians  (many  distinct  jobs  by  equipment  system,  each  with  its  own  advanced  skills 
training  course).  Occupations  with  high  volume  of  trainees  and  complex  tasks  were  analyzed. 

One  occupation  studied  dealt  with  sophisticated  computers  and  switching  equipment  systems 
maintenance.  The  equipment  was  so  complex  that  the  training  and  personnel  were  specialized  by 
the  type  of  equipment  they  maintained.  As  such,  this  occupation  had  a  common  core  but  seven 
different  major  types  of  equipment  systems,  each  with  its  own  training  program  and  jobs  (see 
Figure  2). 
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Figure  2.  Current  Electronic  Computer  and  Switching  Systems  Training  Programs  and 
Jobs. 

Problems  were  beginning  to  show  up  at  a  senior  level;  technicians  were  so  specialized  that 
they  knew  little  about  the  entire  occupation.  Therefore,  when  decisions  were  being  made,  certain 
specialty  areas  may  have  been  underemphasized  due  to  the  lack  of  the  decision  makers  experience 
with  some  equipment. 

TIDES  was  used  to  examine  various  ways  to  collapse  the  seven  separate  tracks  into  a  smaller 
set  of  subspecialties;  the  final  consensus  among  representatives  of  the  various  constituencies  was 
for  a  common  (general)  track  plus  three  specialized  tracks.  The  group  examining  the  problem  (a 
Utilization  &  Training  Workshop)  reached  this  conclusion  by  assessing  the  similarities  between 
the  subspecialties.  It  was  discovered  that  even  though  the  equipment  was  different,  the  differences 
were  fewer  than  the  similarities.  This  finding  suggested  some  simplification  of  the  classification 
structure  and  training  programs  (see  Figure  3). 
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Figure  3.  Proposed  New  Job  and  Training  Structure  for  Electronic  Computer  and 
Switching  Systems  Maintenance  Specialty. 

This  consolidation  reduced  the  overhead  needed  to  manage  the  occupation  and  will  increase 
the  experience  base  for  future  senior  technicians.  Most  importantly,  when  combined  with  other 
Air  Force  personnel  policy  changes,  including  increasing  average  assignment  times  from  32  to  48 
months,  there  was  a  potential  reduction  of  $1.7  million  in  annual  training  costs,  even  though  each 
new  individual  would  receive  more  complete  training  than  in  the;  past. 

Career  Field  Planning  Documents 

In  addition  to  the  TIDES  "what  if  capability,  the  software  also  contains  a  shell  for  developing 
written  guidance  for  specialty  planning  documents.  Such  documents  are  called  Career  Field 
Education  and  Training  Plans  (CFETPs).  CFETPs  identify  normal  career  progression  for 
specialties,  define  the  appropriate  skills  necessary  and  level  of  proficiency  for  each  major  category 
of  skill  courses  available  and  any  resource  constraints  for  future  learning.  These  documents 
usually  take  considerable  time  and  energy  to  compile.  They  must  be  completed  by  SMEs  in  the 
specialty,  typically  four  or  five  people  work  on  it  for  several  days.  Using  the  TIDES  CFETP 
software  shell  (see  Figure  4),  subject  matter  experts  can  produce  a  draft  plan,  often  in  a  single 
day.  The  shell  provides  a  standardized  format,  help  screens,  examples  of  text  which  can  be  used  if 
they  desire  and  general  instructions  on  how  to  complete  the  document. 
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Figure  4.  Career  Field  Education  &  Training  Plan  (CFETP)  Software  Shell 


TIDES  Current  Efforts 

The  TIDES  technology  is  currently  running  on  a  mainframe  computer,  utilizing  flat-file  data 
bases.  It  is  not  particularly  user  friendly  in  its  current  form.  Enhancements  currently  being 
developed  will  make  TIDES  run  on  a  386  or  better  DOS-based  microcomputer.  Additionally, 
the  technology  is  being  made  more  user  friendly  by  utilizing  relational  data  bases.  It  is  envisioned 
that  once  an  initial  TIDES  data  base  for  a  specialty  has  been  developed,  TIDES  users  will  be  able 
to  sit  at  their  personal  computers,  formulate  and  run  various  scenarios  weighing  the  alternatives, 
and  forecast  the  impacts  of  decisions  before  they  become  policy. 

Additional  efforts  are  underway  to  experiment  using  the  TIDES  technology  in  occupations 
utilizing  civilians  as  well  as  military  personnel.  Department  of  Defense  civilians  have  different 
mobility  requirements,  job  descriptions,  and  career  structures  than  military  personnel  performing 
the  same  jobs.  It  is  expected  the  TIDES  can  be  applicable  to  civilian  as  well  as  military 
occupations.  Therefore,  it  is  likely  that  a  decision  support  system  such  as  TIDES  could  be 
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applied  in  large  civilian  organizations  with  high  volume,  expensive  training  prograips.  The  TIDES 
would  be  extremely  useful  in  their  training,  personnel  policy,  and  budgetary  decision  making. 

Portions  of  the  TIDES  technology  may  also  be  applicable  to  Officer  specialties.  One  project 
underway  will  adapt  the  Career  Field  Education  and  Training  Plan  (CFETP)  document  shells  for 
use  with  officer  specialties.  Eventually,  enlisted,  officer  and  civilian  personnel  working  in  a 
particular  occupation  or  fimctional  area  may  all  be  covered  by  a  common  CFETP.  This  was  one 
of  the  long-range  objectives  formulated  as  a  result  of  the  Air  Force  Chief  of  Staffs  Year  of 
Training  Initiative. 

SUMMARY 

The  TIDES  technology  is  a  computer-based  decision  support  system  embodying  a  new 
technology  for  modeling  occupations.  It  is  a  powerful  tool  for  strategic  manpower,  personnel, 
and  training  planning,  and  represents  a  major  advance  in  occupational  analysis  technology  and 
training  requirements  planning.  The  TIDES  provides  significant  new  data  analysis  and  modeling 
for  occupations  including  comprehensive  portrayals  of  existing  jobs  and  training  programs.  The 
TIDES  goes  beyond  simply  describing  the  current  state  of  affairs  to  permit  exploration  of 
hypothetical  scenarios.  Although  the  system  was  designed  primarily  for  use  by  training  decision 
makers,  the  integrated  TIDES  approach  has  considerable  potential  for  promoting  the  eventual 
integration  of  human  resources  management-manpower,  personnel,  training,  and  budgeting  with 
other  human  resources  planning  functions. 
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APPENDIX  A  -  TDS  &  TIDES  TECHNOLOGIES: 
AN  ANNOTATED  BIBLIOGRAPHY  ^ 


Ruck,  H.  W.  (1 982,  Febmary).  Research  and  development  of  a  training  decisions  system.  Proceedings 
of  the  Society  for  Applied  Learning  Technology.  Orlando,  FL. 

A  ccHiceptual  overview  of  a  planned  line  of  research  and  development,  to  explore  the  possibility  of  helping 
Air  Force  managers  vsiio  make  critical  decisions  in  the  area  of  training  to  make  better  decisions,  through  the  use 
of  existing  data  or  by  collecting  new  informaticai.  A  critical  element  of  this  R&D  will  be  an  attempt  to  determine 
the  costs  of  on-the-job  training. 

Vaughan,  D.  S.,  Yadrick,  R.  M.,  Perrin,  B.  M.,  &  Mitchell,  J.  L.  (1985,  May).  Clustering  tasks  into  training 
modules  in  the  Air  Force  training  decisions  system.  Proceedings  of  the  Fifth  International 
Occupational  Analysts  Workshop  (pp.  126-145).  Randolph  AFB,  TX:  USAF  Occupational 
Measurement  Center. 

A  report  of  on-going  efforts  to  develop  methods  for  finding  groups  of  tasks  which  can  be  trained  together; 
i.e.,  have  similar  skill  and  knowledge  requirements,  are  performed  by  the  same  people,  or  share  other  similar 
characteristics.  This  initial  investigation  included  alternative  methods  for  measuring  task  characteristics, 
assessing  similarities  among  tasks,  and  identifying  clusters  of  tasks. 

Garcia,  S.  K.  (1985,  October).  Development  of  an  Air  Force  training  decisions  system.  Proceedings  of 
the  27th  Annual  Conference  of  the  Military  Testing  Association  (pp.  262-264).  San  Diego,  CA: 

Naval  Persoimel  Research  and  Development  Center. 

An  overview  of  to  the  Training  Decisions  System  R&D  and  description  of  the  TDS  subsystems;  Task 
Characteristics  Subsystem  (TCS),  Field  Utilization  Subsystem  (FUS),  Resource/Cost  Subsystem  (RCS),  and 
Integration/Optimization  Subsystem  (lOS). 

Perrin,  B.  M.,  Vaughan,  D.  S.,  Yadrick,  R.  M.,  &  Mtchell,  J.  L.  (1985,  October).  Defining  task  training 
modules:  coperformance  clustering.  Proceedings  of  the  27th  Annual  Conference  of  the  Military 
Testing  Association  (pp.  265-270).  San  Diego,  CA:  Naval  Personnel  Research  and  Development 
Center. 

This  study  compares  coperformance  clustering  of  tasks  with  task  groupings  made  by  groups  of  subject- 
matter  experts  (SMEs),  using  the  Jaccard  and  the  Fowlkes  &  Mallows  statistics  (corrected  versions  of  the  RAND 
statistic).  Results  surest  that  coperformance  clustering  agrees  with  SME  clustering  at  least  as  well  as  one  group 
of  SMEs  agree  with  another  SME  group.  Thus,  this  approach  appears  promising  for  generating  Task  Modules 
(TMs)  which  may  be  useful  in  the  TDS  R&D. 


‘  The  papers  and  presentations  in  Appendices  A  and  B  are  Usted  in  chronological  order.  [As  of:  1 1/95] 
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Yadrick,  R.  M.,  Vaughan,  D.  S.,  Perrin,  B.  M.,  &  Mitchell,  J.  L.  (1985,  October).  Evaluating  task  training  ■ 
modules:  SME  clustering  and  comparisons.  Proceedings  of  the  27th  Annual  Conference  of  the 
Military  Testing  Association  (pp.  271-275).  San  Diego,  Califorma:  Naval  Personnel  Research  and 
Development  Center. 

This  study  builds  on  the  previous  one  by  comparing  coperformance  clusters  with  those  of  SME  groups 
using  different  procedures;  one  group  starting  from  coperformance  piles  of  cards  (one  task  per  card)  and  a  second 
group  starting  from  piles  of  cards  sorted  by  Specialty  Training  Standard  (STS)  paragraph.  Subsequently  the  two 
SME  groups  negotiated  to  consensus  on  how  the  tasks  should  be  group,  which  provides  a  standard  for 
comparison.  Results  suggest  TMs  constructed  by  a  single  group  of  SMEs  may  not  be  stable,  but  perhaps  some 
combination  of  coperformance  clustering  with  SME  review  of  TMs  m^  be  needed. 

Mitchell,  J.  L.,  &  Phalen,  W.  J.  (1985,  October).  Non-hierarchical  clustering  of  Air  Force  jobs  and  tasks. 
Proceedings  of  the  27th  Annual  Conference  of  the  Military  Testing  Association  (pp.  276-281). 

San  Diego,  California:  Naval  Personnel  Research  and  Development  Center. 

In  a  typical  CODAP  study,  5  to  20  percent  of  the  total  cases  may  fall  outside  of  the  identified  job  types. 
An  attempt  was  made  to  use  Non-hierarchical  reclustering  of  job  types  to  refine  the  identified  job  groups  in  a 
study  of  Security  and  Law  Enforcement  personnel  (AFS  81 IXX).  Preliminary  results  strongly  suggest  that  the 
Non-hierarchical  clustering  process  has  considerable  potential  in  refining  the  typical  occupational  analysis 
process,  and  increasing  the  coverage  of  identified  job  types  to  95  percent  of  cases  or  better. 

Mtchell,  J.  L.,  Vaughan,  D.  S.,  Yadrick,  R.  M.,  &  Collins,  D.  L.  (1987,  May).  New  methods  for  portraying 
dynamic  training  and  job  patterns  within  Air  Force  specialties.  Proceedings  of  the  Sixth  International 
Occupational  Analysts'  Workshop  (pp.  103-113).  San  Antonio,  TX:  USAF  Occupational 
Measurement  Center. 

The  typical  occupational  analysis  report  identifies  job  types  based  on  CODAP  hierarchical  clustering 
based  on  the  similarity  of  tasks  performed  and  time  spent  on  those  tasks.  The  job  types  include  cases  with  similar 
profiles  regardless  of  time-in-service  or  time-in-career  field,  which  is  a  rather  static,  two-dimensional  perspective. 
Resorting  job  types  members  by  total  active  federal  months  in  service  (TAFMS)  categories  results  in  a  more 
dynamic  picture  of  job  type  members,  as  they  are  dispersed  over  the  span  of  a  typical  career  (20+  years  of 
service). 

Collins,  D.  L.,  Hernandez,  J.  M,  Ruck,  H.  W.,  Vaughan,  D.  S.,  Mitchell,  J.  L.,  &  Rueter,  F.  H.  (1987, 

August).  Training  decisions  system:  Overview,  design,  and  data  requirements  (AFHRL-TP-87- 
25).  Brooks  AFB,  TX:  Training  Systems  Division,  Air  Force  Human  Resources  Laboratory. 

The  Training  Decisions  System  (TDS)  research  and  development  effort  evolved  from  a  recognition  by  HQ 
ATC  and  USAF/DPPT  that  (a)  independent  decisions  were  being  made  by  different  agencies  within  the 
manpower,  personnel,  and  training  (MPT)  communities;  (b)  there  was  no  way  to  proactively  estimate  what  the 
impact  of  various  training  options  would  be;  and  O  there  were  no  comprehensive,  readily  accessible  cost  data 
regarding  OJT.  MPT  decisions-although  they  are  the  best  possible  given  the  available  information-could  benefit 
significantly  from  an  accurate  data  base  and  modeling  c^ability  oriented  toward  the  macro-level  decision  maker. 
This  report  provides  an  overview  of  the  TDS  R&D  effort  and  the  current  design  of  this  system. 
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Perrin,  B.  M.,  Vaughan,  D.  S.,  Mitchell,  J.  L.,  Collins,  D.  L.,  &  Ruck,  H.  W.  (1987,  October).  Effects  of 
data  collection  format  on  occupational  analysis  task  factor  ratings.  Proceedings  of  the  29th  Annual 
Conference  of  the  Military  Testing  Association  (pp.  96-100).  Ottawa,  Ontario,  Canada:  Directorate 
of  Military  Occupational  Structures,  Canadian  National  Defence  Headquarters. 

This  study  compared  collecting  multiple  task  factor  ratings  on  a  single  response  page  versus  collecting 
the  same  ratings  independently.  The  correlations  among  the  task  factor  ratings  was  systematically  higher 
when  collecting  on  the  same  form  than  when  collected  via  separate  instruments.  These  findings  suggest  that 
task  factor  ratings  should  be  collected  separately  in  both  research  and  operational  settings. 

Ruck,  H.  W.,  &  CoUins,  D.  L.  (1987,  October).  A  microcomputer  simulation  of  an  Air  Force  training 

decisions  system.  Proceedings  of  the  29th  Annual  Conference  of  the  Military  Testing  Association 
(pp.  158-163).  Ottawa,  Ontario,  Canada:  Directorate  of  Mlitary  Occupational  Structures,  Canadian 
National  Defence  Headquarters. 

The  results  of  a  four-year,  AFHRL  exploratory  6.2  R&D  effort  to  develop  a  computer-based  Training 
Decisions  System  (TDS)  are  discussed  as  is  a  microcomputer  simulation  of  the  system.  TDS  displays  include 
modeling  of  the  current  and  alternative  Utilization  &  Training  patterns  as  well  as  allocations  of  training  to  various 
training  settings.  The  simulation  is  a  menu-driven,  self-paced  system  which  provides  users  with  options  to  select 
data  displays  and  possible  optimization  outcomes.  The  examples  of  products  illustrate  the  prototype  system  as 
designed  to  operate  on  the  AFHRL  mainfi'ame  computer  in  a  batch  processing  mode.  The  TDS  is  designed  to 
interface  with  CODAP  and  other  AFHRL  systems,  to  minimize  the  development  of  new  TDS  software. 

Mtchell,  J.  L.,  Ruck,  H.  W.,  &  Driskill,  W.  E.  (1 988).  Task-based  training  program  development.  In  Gael, 

Ed  (Ed.),  The  Job  Analysis  Handbook  for  Business,  Industry,  and  Government  (y  ohxme  1;  pp.205- 
215.  New  York,  NY:  John  Wiley  &  Sons. 

The  TDS  is  mentioned  in  this  overview  chapter  as  one  of  the  outcomes  of  the  task-based  approach  to  job 
analysis,  and  as  a  natural  evolution  of  the  Training  Emphasis  R&D  which  attempted  to  prioritize  occupational 
tasks  in  terms  of  their  priority  for  training. 

Yadrick,  R.  M.,  Knight,  J.  R.,  Mtchell,  J.  L.,  Vaughan,  D.  S.,  &  Perrin,  B.  M.  (1988,  July).  Training 

decisions  system:  Development  of  the  field  utilization  subsystem  (AFHRL-TR-88-7).  Brooks  AFB, 
TX:  Training  Systems  Division,  Air  Force  Human  Resources  Laboratory. 

A  summary  report  of  the  initial  TDS  R&D  project  with  respect  to  the  development  of  various  t)q)es  of 
utilization  and  training  patterns  (U&TPs)  to  "model"  Air  Force  specialties  (AFSs).  The  Field  Utilization 
Subsystem  (FUS)  has  three  major  components:  (1)  the  current  U&TP  component  to  describe  the  AFS,  (2) 
Alternative  U&TP  component  to  possible  changes  in  the  AFS  as  alternative  U&TPs,  and  (3)  the  Management 
Preferences  component  to  collect  and  display  the  opinions  of  various  fimctional  and  training  managers  among 
the  various  U&TPs. 
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Perrin,  B.  M.,  Knight,  J.  R.,  Mitchell,  J.  L.,  Vaughan,  D.  S.,  &  Yadrick,  R.  M.  (1988,  September). 

Training  decisions  system:  Development  of  the  task  characteristics  subsystem  (AFHRL-TR-88-15, 

AD-Al  99  094).  Brooks  AFB,  TX:  Training  Systems  Division,  Air  Force  Human  Resources 

Laboratory. 

This  report  documents  the  research  undertaken  in  initial  development  of  the  Task  Characteristics 
Subsystem  (TCS),  one  of  four  basic  subsystems  of  the  prototype  Training  Decisions  System  (TDS).  The  TCS 
addresses  what  tasks  are  required  to  be  trained  and  where  (training  settings)  to  allocate  those  requirements.  This 
two-component  involves  first  the  construction  of  task  training  modules  (TMs)  fi'om  occupational  survey  (OS) 
data  The  second  component  allocates  TMS  to  various  training  settings,  such  as  formal  classroom  training,  on- 
the-job  training  (OJT),  and  others,  and  determines  levels  of  proficiency  achieved  by  setting.  A  number  of 
methodologies  were  tested  for  the  clustering  of  OS  tasks  into  TMS.  Some  TMS  were  also  subjected  to  task 
analysis  to  assess  homogeneity  of  required  skills  and  knowledge.  New  Comprehensive  Occupational  Data 
Analysis  Programs  (COD^)  routines  were  developed  and  have  been  implemented.  Three  measures  of  T.M.-by- 
setting  proficiency  were  pilot  tested.  Guidelines  for  applying  procedures  developed  in  this  R&D  were  also 
developed. 

Mtchell,  J.  L.,  Vaughan,  D.  S.,  Yadrick,  R.  M.,  CoUins,  D.  L.,  &  Hernandez,  J.  M.  (1988,  September). 

The  Air  Force  training  decisions  system:  Modeling  job  and  training  flows  (AFHRL-TP-88-12). 

Brooks  AFB,  TX:  Training  Systems  Division,  Air  Force  Human  Resources  Laboratory. 

The  Training  Decisions  System  (TDS)  uses  information  about  clusters  of  related  tasks  (Task  Training 
Modules  or  TMS),  a  dynamic  model  of  specialty  job  flows  and  training  programs,  and  data  on  unit  training 
capacities  and  costs,  to  compare  job  and  training  options.  In  the  prototype  TDS  development,  graphic  and 
mathematical  models  of  four  Air  Force  specialties  and  several  alternative  Utilization  and  Training  (U&T)  Patterns 
were  created.  Managers  preferences  for  the  various  alternatives  were  collected  through  a  survey  questionnaire. 
Results  were  highly  favorable;  analysis  indicated  a  high  degree  of  agreement  among  raters.  A  post  hoc  analysis 
suggested  some  systematic  differences  of  opinion  among  subgroups  of  raters.  Managers  evaluated  the  project 
very  favorably  and  indicated  the  models  had  successfully  captured  and  communicated  the  U&T  Patterns  of  the 
four  occupations. 

Vaughan,  D.  S.  (1988,  November).  Overview  of  the  training  decisions  system:  Results  and  products.  In 

the  symposium.  The  Air  Force  Training  Decisions  System:  R&D  Results  (W.  R.  Beimett,  Chair). 

Proceedings  of  the  30th  Annual  Conference  of  the  Military  Testing  Association  (pp.  779-784). 

Arhngton,  Virginia:  Army  Research  Institute. 

Overview  of  the  technical  approach  and  significant  outcomes  of  the  TDS  R&D  at  the  conclusion  of  the 
initial  R&D  effort.  The  approach  used  in  TDS  is  to  model  each  problem  or  potential  solution  as  an  independent 
Utilization  and  Training  Pattern  (U&TP);  once  a  problem  or  potential  solution  has  been  identified,  it  is  expressed 
in  terms  or  quantities  of  the  TCS  (i.e.,  task  modules,  training  settings,  and  allocation  curves).  The  FUS  modeling 
c^ability  is  used  to  translate  the  potential  solution  into  modifications  of  the  U&TP  data  base  for  the  AFS;  that 
is  as  an  alternative  to  the  current  U&TP  files.  FUS  and  RCS  software  are  employed  to  generate  products  for  each 
alternative  and  results,  in  the  form  of  training  costs  and  capacities  are  compared  to  basehne  data  by  Air  Force 
decision  makers. 
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Perrin,  B.  M.  &  J.  R.  Knight  (1988,  November).  The  task  characteristics  subsystem:  Allocating  task 
modules  to  training  settings.  In  the  symposium.  The  Air  Force  Training  Decisions  System:  R&D 
Results  (W.  R.  Bennett,  Chair).  Proceedings  of  the  30th  Annual  Conference  of  the  Military  Testing 
Association  (pp.  785-790).  ArUngton,  Virginia:  Army  Rese'arch  Institute. 

The  Task  Characteristics  Subsystem  (TCS)  provides  data  on  training  setting  allocations  and  on  task 
training  modules  (IMS).  TMS  are  the  basic  building  blocks  of  the  TDS,  in  that  all  training  courses  and  jobs  are 
described  in  terms  of  TMS,  and  training  costs  are  determined  using  T.M. -based  procedures.  Procedures  for 
deriving  TMS  are  described,  as  are  data  collection  methods  for  deriving  equations  for  relating  training  hours  to 
required  proficiency.  Such  functions  (curves)  permit  examination  of  the  possibihties  of  training  in  various 
training  setting  configurations,  and  for  estimating  on-the-job  training  (OJT)  costs. 

Mitchell,  J.  L.,  &  Yadrick,  R.  M.  (1988,  November).  The  field  utihzation  subsystem:  Job  and  training 
pattern  simulations.  In  the  symposium.  The  Air  Force  Training  Decisions  System:  R&D  Results 
(W.  R.  Bennett,  Chair).  Proceedings  of  the  30th  Annual  Conference  of  the  Military  Testing 
Association  (pp.  791-796).  ArUngton,  Virginia:  Army  Research  Institute. 

The  Field  Utihzation  Subsystem  (FUS)  provides  information  for  defining  training  and  job  assignment 
patterns,  as  weU  as  informaticai  on  management  preferences  for  the  current  and  plausible  alternative  approaches 
to  training,  assigning,  and  using  airmen  in  a  particular  Air  Force  Specialty  (AFS)  over  the  span  of  their  Air  Force 
careers.  The  FUS  consists  of  a  Current  U&T  Pattern  component,  to  describe  the  existing  specialty,  an 
Alternative  U&T  Pattern  component  to  hypothesize  one  or  more  alternative  configurations  of  jobs  and  training 
programs,  and  an  FUS  Flow  Simulation  prc^am  which  quantifies  the  numbers  of  people  involved  on  an  annual 
basis. 

Rueter,  F.  H.  &  Feldsott,  S.  (1988,  November).  Resource/cost  subsystem:  Estimating  training  capacities 
and  costs.  In  the  symposium.  The  Air  Force  Training  Decisions  System:  R&D  Results  (W.  R. 

Bennett,  Chair).  Proceedings  of  the  30th  Annual  Conference  of  the  Military  Testing  Association 
(pp.  797-802).  ArUngton,  Virginia:  Army  Research  Institute. 

The  Resource\Cost  Subsystem  (RCS)  was  developed  to  prove  three  distinct,  yet  interrelated  analytic 
c^abiUties:  (1)  determination  of  the  types  and  amounts  of  resources  required  to  provide  training  on  each  T.M. 
in  each  training  setting,  and  estimate  the  amounts  of  those  resources  available  for  use  at  typical  sites;  (2)  assess 
the  edacities  of  sites  to  accommodate  different  vdumes  of  training  on  different  combinations  of  TMS  in  different 
training  states,  and  (3)  estimation  of  the  variable  costs  that  must  be  incurred  in  providing  training  on  each  T.M. 
in  each  training  setting,  and  in  providing  particular  volumes  of  training  in  specific  training  states. 

Vaughan,  D.  S.,  &Eschenbreimer,  A.  J.  (1988,  November).  Integration/optimization  subsystem:  An 

integrated  modeUng  approach.  In  the  symposium.  The  Air  Force  Training  Decisions  System:  R&D 
Results  (W.  R.  Bennett,  Chair).  Proceedings  of  the  30th  Annual  Conference  of  the  Military  Testing 
Association  (pp.  803-808).  ArUngton,  Virginia:  Army  Research  Institute. 

The  Integration/Optimization  Subsystem  (lOS)  ties  together  the  other  three  TDS  subsystems  into  one 
overall  functional  system.  The  interconnections  of  the  subsystems  provide  the  capacity  to  optimize  measures 
derived  fi'om  one  subsystem  relative  to  constraints  obtained  fi'om  the  others,  and  simultaneously  process  data 
files  derived  fi'om  different  subsystems.  The  lOS  also  provides  the  interface  with  users;  that  is,  the  subsystem 
receives  all  requests,  caUs  appropriate  data  fi'om  the  other  subsystems  or  TDS  files,  and  creates  products  to  meet 
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the  users'  needs.  In  all  of  its  functions,  the  lOS  governs  the  interaction  among  the  TDS  subsystems  and  various 
data  sources,  and  the  relationship  of  the  system  with  various  types  of  users. 

Ruck,  H.  W.  (1988,  November).  Discussion  of  the  TDS  project.  In  the  symposium.  The  Air  Force 

Training  Decisions  System:  R&D  Results  (W.  R.  Beimett,  Chair).  Proceedings  of  the  30th  Annual 
Conference  of  the  Military  Testing  Association  (page  809),  Arlington,  Virginia:  Army  Research 
Institute. 

The  development  of  the  Training  Decisions  System  represents  the  culmination  of  a  line  of  research  which 
began  in  the  mid-1970's,  and  which  has  progressed  step-by-step  to  the  present  evolved,  computer-based,  decision 
support  system.  The  TDS  is  designed  primarily  to  provide  Air  Force  decision  makers  with  information  in  a  form 
which  will  assist  them  in  making  critical  decisions  involving  training.  The  operational  concept  involved  is  to  be 
able  to  show  decision  makers  some  of  the  possible  consequences  of  their  decisions  in  specific  quantitative  terms, 
something  which  has  never  been  done  before.  The  very  complex  TDS  challenge  involved  designing  new  methods 
to  collect,  aggregate,  predict,  and  displ^  Air  Force  specialty  job  and  training  data,  including  modehng  alternative 
specialty  patterns  and  estimation  of  capacities  and  costs  for  formal  courses  as  well  as  on-the-job  training  (OJT). 


Vaughan,  D.  S.,  Mitchell,  J.  L.,  Yadrick,  R.  M.,  Perrin,  B.  M.,  Knight,  J.  R.,  Eschenbrenner,  A.  I,  Rueter, 

F.  H.,  &  Feldsott,  S.  (1989,  June).  Research  and  Development  of  the  Training  Decisions  System 
(AFHRL-TR-88-50).  Brooks  AFB,  TX:  Training  Systems  Division,  Air  Force  Human  Resources 
Laboratory. 

This  document  summarizes  the  R&D  undertaken  to  develop  the  Training  Decisions  System  (TDS).  The 
TDS  is  a  computer-based  decision  aid  to  be  used  in  planning  the  what  (training  content),  the  where  (technical 
school.  Field  Training  Detachment,  on-the-job  training,  or  self-stucfy),  and  the  when  (at  what  point  in  an  airman's 
career)  of  training  required  for  an  Air  Force  specialty.  Further,  the  TDS  incorporates  optimization  strategies  to 
allow  training  managers  to  as  "what  if  questions  related  to  current  and  possible  future  pohcy  changes  within 
the  Air  Force  training  environment.  In  addition,  this  report  contains  a  brief  conceptual  overview  of  the  three 
major  data-based  subsystems  and  the  fourth  integrating/optimization  subsystem  which  compose  the  present 
TDS.  Implications  fi'om  recent  trends  in  Air  Force  personnel  and  training  policy  that  affect  training  management 
and  planning  and  a  review  of  agencies  within  the  Air  Force  which  could  support  and  use  technologies  such  as 
the  TDS  are  discussed.  Finally,  conclusions  and  recommendations  for  using  TDS  products  in  the  operational  Air 
Force,  including  a  draft  Air  Force  Regulation  for  implementing  and  maintaining  the  TDS,  are  presented. 

Vaughan,  D.  S.,  &  Lamb,  T.  A.  (1989,  November).  Training  decisions  modeling  outcomes:  Conclusions  and 
imphcations.  In  the  Symposium  (Ruck,  H.  W.,  Chair),  The  Air  Force  training  decision  system: 
Modehng  management  and  pohcy  issues  for  effective  training.  Proceedings  of  the  3 1st  Annual 
Conference  of  the  Military  Testing  Association  (pp.  588-593).  San  Antonio,  TX:  Air  Force  Human 
Resources  Laboratory  and  the  USAF  Occupational  Measurement  Center. 

This  presentation  reviews  and  summarizes  the  Training  Decisions  System  (TDS)  analysis  results  fi'om  the 
previous  presentations.  The  analysis  process  used  to  obtain  these  results  is  also  reviewed.  Overall  conclusions 
are  presented  concerning  the  manpower,  personnel,  and  trainmg  (MPT)  pohcy  issue  that  was  the  subject  of  those 
TDS  analyses-a  25%  reduction  in  travel-to-school  (TDY-to-school)  budgets.  The  TDS  can  also  be  used  to 
examine  many  other  types  of  training  structure-  related  issues,  as  well  as  MPT-related  issues.  This  wide  span 
of  issues  that  can  be  addressed  and  analyzed  using  TDS  technology  are  described,  and  some  examples  are 
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provided.  Finally,  overall  status  and  &ture  directions  for  training  decisions  support  research  and  application  are 
discussed. 

Knight,  J.  R.,  &  Bennett,  W.  R.  (1989,  November).  Utihzation  &  training  pattern  simulation  results.  In 

the  Symposium  (Ruck,  H.  W.,  Chair),  The  Air  Force  training  decision  system:  Modehng  management 
and  policy  issues  for  effective  training.  Proceedings  of  the  31st  Annual  Conference  of  the  Military 
Testing  Association  (pp.  570-575).  San  Antonio,  TX;  Ar  Force  Human  Resources  Laboratory  and 
the  USAF  Occupational  Measurement  Center. 

To  adequately  model  the  flow  of  personnel  through  jobs  and  training  programs,  TDS  employs  a  dynamic 
simulation  approach,  which  gives  an  analyst  the  capability  to  vary  specified  values  (i.e.,  number  entering, 
assignment  probabilities,  attrition  rates,  etc.).  Using  the  U&T  Pattern  Simulation  (UTPSIM)  program,  a  change 
such  as  a  25%  reduction  in  TDY-to-School  budget  can  be  modeled  by  reducing  the  probabihty  of  attending  Field 
Training  Detachment  (FTD)  or  Professional  Military  Education  (PME)  courses  requiring  travel.  The  UTPSIM 
outputs  become,  in  effect,  an  Atemative  U&T  pattern  which  models  the  proposed  change  on  specialty  jobs  and 
training  flows. 

Mitchell,  J.  L.,  &  Lamb,  T.  A.  (1989,  November).  Quantification  of  specialty  training  requirements.  In 
the  symposium.  The  Ar  Force  Training  Decisions  System:  Modeling  Management  and  Policy 
Issues  for  Efficient  Training.  Proceedings  of  the  31st  Annual  Conference  of  the  Military  Testing 
Association  (pp.  576-581).  San  Antonio,  TX:  Ar  Force  Human  Resources  Laboratory  and  USAF 
Occupational  Measurement  Center. 

U&T  pattern  simulation  outputs  are  used  to  calculate  the  total  amount  of  training  required  on  those  task 
modules  each  person  must  learn  to  be  fiilly  proficient  in  his  or  her  job.  By  subtracting  previous  training  received 
in  technical  school,  field  training,  or  self-study  fi'om  the  total  proficiency  required,  the  Training  Proficiency 
(TRNPRF)  program  estimates  the  on-the-job  training  (OJT)  hours  needed  on  all  relevant  task  modules  for  each 
individual  to  reach  the  required  proficiency.  Proposed  changes  in  specialty  jobs  or  training  programs  are  modeled 
in  terms  of  changes  in  various  training  requirements.  TRNPRF  is  used  to  quantify  the  impact  of  changes  on  the 
total  annual  training  requirements  of  the  specialty  being  studied.  TRNPRF  results  are  then  used  as  input  data 
for  the  TDS  Resource/Cost  Subsystem  (RCS)  cost  and  capacity  programs. 

Rueter,  F.  H.,  &  Feldsott,  S.  (1989,  November).  Training  capacities  and  costs  estimation  for  328X4.  In  the 
Symposium  (Ruck,  H.  W.,  Chair),  The  Ar  Force  training  decision  system:  Modehng  management 
and  poUcy  issues  for  effective  training.  Proceedings  of  the  31st  Annual  Conference  of  the  Military 
Testing  Association  (pp.  582-587).  San  Antonio,  TX:  Ar  Force  Human  Resources  Laboratory  and 
the  USAF  Occupational  Measurement  Center. 

The  Resource/Cost  Subsystem  (RCS)  of  the  TDS  provides  capabihties  to  estimate  the  variable  costs  of 
providing  the  training  associated  with  a  U&T  pattern,  and  to  evaluate  the  capacities  of  different  types  of  Ar 
Force  units  to  accommodate  the  amounts  of  training  needed  within  that  U&T  pattern.  Using  the  RCS,  the 
impacts  on  cost  and  c^adty  that  result  fi'om  a  change  such  as  a  25  percent  reduction  in  TDY-to-school  funding 
can  be  assessed  by  comparing  the  cost  and  capacity  estimates  developed  for  the  current  U&T  patterns  and  an 
alternative  U&T  pattern  that  models  the  proposed  change  in  training  flows. 
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Vaughan,  D.  S.,  &  Bennett,  W.  R.  (1989,  November).  Overview  of  the  training  decisions  system: 

Management  and  pohcy  issue  development.  In  the  S3anposium  (Ruck,  H.  W.,  Chair),  The  Air  Force 
training  decision  system;  Modehng  management  and  pohcy  issues  for  effective  training.  Proceedings 
of  the  31st  Annual  Conference  of  the  Military  Testing  Association  (pp.  564-569).  San  Antonio,  TX; 
Air  Force  Human  Resources  Laboratory  and  the  USAF  Occupational  Measurement  Center. 

The  Air  Force  Training  Decisions  System  (TDS)  is  a  computer-based  decision  support  system  for  use  by 
Air  Force  manpower,  persoimel  and  training  (MPT)  decision  makers  to  evaluate  training  impact  of  various  MPT 
policy  options.  In  particular,  tiie  TDS  models  the  flow  of  airmen  through  jobs  in  an  Air  Force  occupation, 
estimates  formal  and  on-the-job  training  quantities  required  to  support  job  requirements,  costs,  and  capacities. 
Examples  of  issues  which  can  be  modeled  with  this  technology  are  described  and  evaluated. 

Rueter,  F.  H.,  Feldsott,  S.  &  Vaughan,  D.  S.  (1989,  December).  Training  decisions  system:  Development 
of  the  resource  cost  subsystem  (AFHRL-TR-88-52).  Brooks  AFB,  TX:  Training  Systems  Division, 

Air  Force  Human  Resources  Laboratory. 

This  report  contains  a  brief  conceptual  overview  of  the  interrelationships  between  the  Resource  Cost 
Subsystem  (RCS)  and  the  three  other  major  subsystems  which  compose  the  present  Training  Decisions  System 
(TDS).  The  report  highlights  the  novel  approach  undertaken  and  exphcates  the  econometric  and  mathematical 
assumptions  underlying  the  development  of  the  model  within  this  subsystem,  including  the  cost  analysis  and 
capaci^  estimation  capabihties  which  have  been  built  into  the  RCS.  These  capabihties  include  methods  for 
identifying  training  resource  requirements,  highhghting  constraints  that  impede  the  provision  of  training  at 
representative  Air  Force  sites,  and  calculating  variable  costs  for  the  provision  of  training  for  various  training 
settings  at  representative  sites.  This  report  also  summarizes  data  sources  identified  and  the  data  collection 
activities  performed  and  provides  sample  data  collection  instruments  developed  to  obtain  information  for  the 
RCS.  In  addition,  the  repeat  examines  implications  fi'om  recent  trends  in  Air  Force  personnel  and  training  pohcy 
for  training  cost  analysis.  Finally,  training  resource  cost  and  capacity  issues  are  identified  and  recommendations 
fOT  future  RCS-related  research  activities  and  for  using  RCS  products  in  the  operational  Air  Force  are  proposed 
and  discussed. 

Lamb,  T.  A.,  Hernandez,  J.  M.  M.,  &  Villanueva,  T.  (1 989).  Task  clustering  methodology  comparison 
(AFHRL-TP-88-68).  Brooks  AFB,  TX:  Training  Systems  Division,  Air  Force  Human  Resources 
Laboratory. 

This  report  summarizes  experimental  work  comparing  task  clustering  methodologies  in  an  attempt  to 
validate  the  TDS  task  co-performance  plus  SME  review  and  refinement  procedures.  While  a  number  of 
conditions  and  variables  were  involved,  the  report  concludes  that  the  task  coperformance  clustering  used  in  TDS 
does  at  least  as  well  as  other  methodologies  for  grouping  tasks  which  probably  share  an  underlying  set  of 
common  skills  and  knowledges. 

Knight,  J.  R.,  Vaughan,  D.  S.,  Yadrick,  R.  M.,  &  Bennett,  W.  R.  (1 990,  April).  Modeling  career  paths  in  Air 
Force  enhsted  specialties.  Proceedings  of  the  Twelfth  Psychology  in  the  DoD  Symposium  (pp.  250- 
254).  Colorado  Springs,  CO:  USAF  Academy,  Department  of  Behavioral  Sciences  and  Leadership. 

Present  mihtary  occupational  analysis  methodologies  focus  on  the  identification  of  job  types  within  an 
occupation  and  analysis  and  evaluation  of  initial  skills  training  programs.  New  methods  have  recently  been 
developed  which  help  identify  career  paths  including  job  assignment  probabihties  and  associated  training 
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required  for  job  mastery.  By  integrating  Job  and  Training  History  survey  information  with  job  analysis  data,  a 
more  comprehensive  model  of  occupational  career  paths  can  be  developed.  Such  d3mamic  models  of  specialties 
represent  a  significant  enhancement  of  current  job  analysis  technology. 

Chin,  K.  B.  O.,  Lamb,  T.  A.,  Vaughan,  D.  S.,  Bermett,  W.  R.  (1990).  Training  Decisions  Modehng 

Technologies.  Proceedings  of  the  Technology  and  Innovations  in  Training  and  Education  (TITE) 
Annual  Conference.  Colorado  Springs,  CO. 

The  Training  Decisions  Modehng  Technology  (TDMT)  involves  a  set  of  basehne  technologies  for 
gathering  and  using  information  about  jobs  performed  by  airmen,  persoimel  assignment  flows,  course  training 
content,  and  training  resources  to  determine  training  edacities  of  representative  units  and  the  most  cost-eflfective 
training  options  available.  The  technology  develops  a  model  of  a  job  specialty's  Utihzation  and  Training  (U&T) 
pattern  according  to  data  collection  results  and  simulates  the  flow  of  airmen  through  it.  Based  on  this  simulation, 
the  system  tabulates  the  costs  associated  with  training  and  evaluates  the  training  capacity  of  the  training  system 
(i.e.,  the  ability  of  the  training  system  to  provide  training  to  a  given  number  of  airmen).  Such  information  should 
be  extremely  useful  for  Air  Force  decision  makers,  by  providing  a  perspective  on  the  potential  outcomes  of  their 
decisions. 

Vaughan,  D.  S.,  Mitchell,  J.  L.,  Knight,  J.  R.,  Bennett,  W.  R.,  &  Buckenmyer,  D.  V.  (1990,  November). 
Developing  a  training  time  &  proficiency  model  for  estimating  Air  Force  specialty  training 
requirements  of  new  weapon  systems.  Proceedings  of  the  32nd  Annual  Conference  of  the  Military 
Testing  Association  (pp.  1 16-121).  Orange  Beach,  AL:  Naval  Education  and  Training  Program 
Management  Support  Activity. 

Estimating  training  costs  and  training  c^adty  constraints  are  among  the  major  manpower,  personnel,  and 
training  issues  in  the  development  of  new  weapon  systems.  Use  of  the  recently  developed  Training  Decisions 
Modeling  Technology  in  the  systems  acquisition  process  is  problematic  since  no  occupational  survey  data  will 
be  available  as  a  basis  for  modehng  the  specialty,  its  jobs,  and  its  training.  This  paper  reports  an  iimovative 
experimental  approach  using  subject  matter  experts'  ratings  of  generic  skill  and  knowledge  categories  for  the 
anticipated  work  to  predict  training  time  and  proficiency  (training  setting-specific  learning  curves).  Regression 
analysis  indicates  that  substantial  proportions  of  the  variance  in  training  time  curves  can  be  predicted  fi'om  such 
ratings.  This  approach  may  improve  training  decision  making  and  logistic  support  analyses  early  in  the  new 
weapon  system  acquisition  process. 

Mitchell,  J.  L.,  Knight,  J.  R.,  Buckenmyer,  D.  V.,  &  Hand,  D.  K.  (1 991,  May).  The  use  of  CODAP  in  the 
Training  Decisions  System.  Proceedings  of  the  Seventh  International  Occupational  Analysts' 
Workshop  (pp.  9-18).  San  Antonio,  TX:  USAF  Occupational  Measurement  Center. 

CODAP  data  is  used  as  the  starting  point  for  studies  of  Air  Force  Specialties  (AFSs)  in  the  Training 
Decisions  System  (IDS).  The  CODAP  task  clustering  algorithm  is  used  to  develop  task  modules  (TMs)  for  the 
AFS,  and  job  types  are  described  using  summary  task  data  at  the  TM  level.  The  MODULES  program  permits 
automated  calculation  of  statistical  indices  for  TMs.  TMs  are  also  used  as  the  basis  for  collecting  additional  data 
with  which  aimual  training  requirements  can  be  quantified  and  training  costs  and  unit  training  capacities  can 
be  estimated,  including  the  annual  costs  of  AFS  on-the-job  training  (OJT)  programs.  By  using  automated 
advanced  CODAP  analysis  programs,  the  initial  phases  of  a  IDS  study  can  be  accomphshed  more  cost  effectively 
and  expeditiously.  In  the  long-term,  the  use  of  a  common  data  base  (CODAP  task  data)  facihtates  integration 
with  other  task-based  systems,  such  as  the  Advanced  OJT  System  (AOTS;  now  the  Base-level  Training  System 
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or  BTS).  Such  integration  will  also  facilitate  feedback  to  the  USAFOM  Squadron  to  enhance  future  task  hst 
development  and  improve  occupational  analysis  processes. 

Mitchell,  J.  L.,  Vaughan,  D.  S.,  Knight,  J.  R.,  Buckenmyer,  D.  V.,  <&  Bennett,  W.  R.  (1991,  May).  An 
operational  test  of  the  training  decisions  system.  Proceedings  of  the  Seventh  International 
Occupational  Analysts'  Workshop  (pp.  274-280).  San  Antonio,  TX;  USAF  Occupational 
Measurement  Center. 

An  operational  test  of  the  Training  Decisions  System  (TDS)  is  underway  in  a  joint  study  of  the  Electronic 
Ccxnputer  and  Switching  Systems  (AFS  305X4)  specialty  with  the  USAF  Occupational  Measurement  Squadron 
of  Air  Training  Command  (ATC).  A  brief  Training  and  Job  History  Survey,  developed  by  the  TDS  R&D  team, 
is  being  administered  with  the  USAF  Job  Inventory.  Other  TDS  data  will  be  gathered  after  the  initial  case  and 
task  clusters  have  been  interpreted.  TDS  software  is  being  loaded  and  tested  on  the  USAFOM  Sq  computer  to 
facilitate  analysis  of  the  AFS  305X4  data  and  to  demonstrate  the  general  utiUty  of  the  system.  By  working 
closely  with  the  USAFOM  Sq  during  the  parallel  analyses  of  this  specialty,  the  IDS  Research  &  Development 
(R&D)  team  should  be  able  to  receive  real-time  feedback  on  the  quahty  and  utihty  of  the  TDS  analysis,  and  can 
anticipate  and  resolve  any  problems  encountered  in  the  eventual  operational  implementation  of  the  system. 

Mitchell,  J.  L.,  Buckenmyer,  D.  V.,  &  Hugely,  H.  (1 991 ,  October).  Development  of  an  integrated  data 
base  for  the  electronic  computer  and  switching  systems  specialty  (AFS  305X4).  Proceedings  of 
the  33d  Annual  Conference  of  the  Military  Testing  Association  (pp.  275-280).  San  Antonio,  TX: 
Human  Resources  Ehrectorate,  Armstrong  Laboratoiy  and  the  USAF  Occupational  Measurement 
Squadron. 

A  joint  study  of  the  Electronic  Computer  and  Switching  Systems  (AFS  305X4)  specialty  is  underway 
through  the  cooperation  of  the  USAF  Occupational  Measurement  Squadron,  Headquarters  Air  Training 
Command,  and  the  Technical  Training  Research  Division  of  the  Armstrong  Laboratory.  A  brief  Training  and 
Job  Histoiy  Survey,  developed  by  the  Training  Decision  System  (TDS)  R&D  team,  was  administered  along  with 
tile  normal  USAFOM  Sq  Job  Inventcsy.  Other  TDS  data  were  gathered  subsequent  to  the  interpretation  of  initial 
case  and  task  clusters,  using  a  common  data  set  on  the  USAFOM  Sq  computer.  TDS  software  was  loaded  and 
tested  on  the  USAFOM  Sq  computer  to  facilitate  this  analysis,  and  to  demonstrate  the  general  utihty  of  the 
system.  The  parallel  occupational  analysis  and  TDS  analysis  projects  are  both  aimed  at  providing  the  AFS 
305X4  Functional  Manager  and  UtiUzation  and  Training  Workshop  participants  with  a  comprehensive 
perspective  on  the  specialty  as  well  as  the  capabihty  to  assess  the  potential  impacts  of  their  decisions  in  terms 
of  total  training  costs  and  AFS  training  edacity.  By  having  the  operational  and  R&D  organizations  work  closely 
together  to  meet  the  real-world  needs  of  the  functional  managers,  this  project  should  identify  and  resolve  any 
issues  critical  to  the  possible  operational  implementation  of  the  TDS  technology. 

Bennett,  W.  R.,  Ruck,  H.  W.,  Mitchell,  J.  L.,  &  Vaughan,  D.  S.  (1991,  October).  Training  and 
organizational  development  in  military  organizations:  impheations  for  diagnosis  and  needs 
assessment.  Proceedings  of  the  33d  Annual  Conference  of  the  Military  Testing  Association  (pp. 
670-675).  San  Antonio,  TX:  Human  Resources  Directorate,  Armstrong  Laboratory  and  the  USAF 
Occupational  Measurement  Squadron. 

Military  organizations  are,  by  definition,  dynamic  organizations.  The  demands  placed  upon  them  are 
unique  and  the  requirements  for  performance  are  crucial  for  national  defense.  In  the  current  fiscal  environment, 
military  organizations  are  experiencing  unparalleled  changes  in  personnel,  mission,  and  structure.  In  this  type 
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of  environment,  the  capability  to  adequately  diagnose  organizational  needs,  and  develop  training  methods  to 
compensate  for  changes  to  the  force  structure  is  critical  to  maintaining  a  qualified  force.  This  paper  examines 
training  as  a  strategy  in  organizational  development.  A  systems  approach  model,  based  on  levels  of  analysis,  for 
the  design  of  training  and  the  identification  of  training  and  organizational  objectives  are  discussed.  The  utility 
of  a  systems  approach  to  organizational  diagnosis  and  training  development,  implementation,  evaluation, 
feedback,  and  refinement  for  continued  organizational  change  in  mihtary  organizations  is  critically  evaluated. 
Finally,  potential  areas  for  future  research  within  military  contexts  are  highhghted. 

Vaughan,  D.  S.,  Perrin,  B.  M.,  Ruck,  H.  W.,  &  Bennett,  W.  R.  (1991,  October).  Estimating  learning 

curves  fi'om  job  analysis  data.  Proceedings  of  the  33d  Annual  Conference  of  the  Military  Testing 
Association  (pp.  269-274).  San  Antonio,  TX:  Human  Resources  Directorate,  Armstrong 
Laboratory  and  the  USAF  Occupational  Measurement  Squadron. 

In  the  Air  Force,  occupational  survey  job  analysis  data  are  very  usefirl  for  the  fi’ont  end  analysis  steps 
in  the  Instructional  Systems  Design  (ISD)  process.  In  particular,  the  Recommended  Training  Emphasis  task 
factor  provides  an  effective  means  of  prioritizing  tasks  for  training  of  first-enlistment  airmen.  However,  these 
data  do  not  provide  direct  information  concerning  the  best  training  settings  (e.g.,  classroom,  self-study, 
hands-on,  on-the-job)  for  training  on  particular  tasks.  We  tested  an  occupational  survey-based  approach  for 
estimating  training  setting-specific  learning  curves  of  task  groups.  Results  showed  that  the  survey  responses 
were  equivalent  in  reliabiUty  to  those  of  conventional  task  factors  and  that  the  learning  curve  models  fit  the 
survey  data  well.  These  learning  curves  provide  useful  information  concerning  the  best  training  settings  for 
particular  task  groups.  Furthermore,  these  curves  provide  a  basis  for  estimating  on-the-job  training 
quantities,  training  costs  and  resource  requirements  in  the  Training  Decisions  System. 

Glushko,  R.  J.  (1991).  Training  decisions  modeling  simulation  environment:  Design  plan  (AL-CR-1991- 
01).  Brooks  AFB,  TX:  Training  Research  Division,  Armstrong  Laboratory,  Human  Resources 
Directorate. 

Air  Force  training  planners  must  determine  the  most  effective  and  efficient  ways  to  train  people  in  the 
various  Air  Force  Specialties  (AFSs)  using  formal  training  courses,  on-the-job  training,  mobile  training, 
contractor  courses,  and  various  other  training  options.  A  Training  Decisions  Modehng  methodology  is  being 
developed  by  the  Air  Force  Human  Resources  Laboratory  (now  Armstrong  Laboratory,  Hupian  Resources 
Directorate)  to  provide  a  more  integrated  and  unified  ^proach  for  making  training  planning  decisions.  The  multi¬ 
year  research  and  development  effort  led  by  McDonnell  Douglas  to  design  and  build  ffie  Training  Decisions 
System  (TDS)  was  the  initial  step  in  bringing  together  different  information  and  trade-off  requirements  into  an 
integrated  decision  support  system.  Search  Technology's  current  project  complements  and  builds  upon  the 
McDonnell  Douglas  TDS  work.  The  project  involves  designing  a  highly  interactive  demonstration  system  that 
will  embo^  more  of  the  evolving  capabilities  of  the  Training  Decisions  Modeling  Methodology  (TDMM).  This 
effort  makes  the  visions  and  benefits  of  the  TDMM  more  apparent  to  potential  users  and  program  supporters. 

Vaughan,  D.  S.,  Knight,  J.  R.,  Mitchell,  J.  L.,  Marshall,  G.  A.,  Feldsott,  S.,  Rueter,  F.  H.,  Hand,  D.  K.,  & 
Ha5mes,  W.  R.  (1991,  April).  Training  Decisions  System;  Software  User's  Manual  (AL-CR-1991- 
0002).  Brooks  AFB,  TX:  Training  Systems  Division,  Armstrong  Laboratory,  Human  Resources 
Directorate. 

A  comprehensive  description  and  users  guide  for  the  Training  Decision  System  (TDS)  software.  This 
proof-of-concept  system  was  developed  in  FORTRAN  77  since  none  of  the  existing  commercial  simulation 
software  coild  provide  the  capacity  for  large  numbers  of  variables  and  complex  assignment  probabilities.  Each 
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TDS  data  file  is  described  in  terms  of  format  and  content;  file  naming  conventions  are  consistent  with  other 
UNISYS  files,  such  as  the  Comprehensive  Occupational  Data  An^sis  Programs  or  CODAP,  since  some  TDS 
files  are  generated  directly  fi-om  CODAP  files.  [Equivalent  file  and  program  names  on  VAX  systems,  used  in 
the  software  development  effort,  are  provided  in  and  appendix]. 


Chin,  K.  B.  O.,  Lamb,  T.  A.,  Bennett,  W.  R.,  &  Vaughan,  D.  S.  (1992,  April).  Introduction  to  training 
decisions  modeling  technologies:  The  training  decision  system  (AL-TP- 1992-00 14).  Brooks  AFB, 
TX:  Technical  Training  Research  Division,  Armstrong  Laboratory,  Human  Resources  Directorate. 

An  introduction  to  the  Training  Decisions  Modehng  Technologies  under  development  at  the  Armstrong 
Laboratory.  These  technologies  are  intended  to  support  Air  Force  Training  Managers  by  providing  them  with 
cost  and  resource  information  about  the  impact  of  their  decisions  on  the  Air  Force  training  system.  The  Training 
Decisicms  System  (TDS),  which  forms  the  baseline  technologies  for  the  research  program,  uses  information  about 
jobs  performed  by  airmen,  personnel  assignment  flows,  course  training  content,  and  training  resources  to 
determine  training  capacities  of  representative  units  and  the  most  cost-effective  training  options  available.  The 
TDS  develops  a  model  of  a  job  specialty's  Utilization  and  Training  (U&T)  pattern  according  to  data  collection 
results  and  gimiilatas  the  flow  of  airmen  through  it.  Based  on  this  simulation,  TDS  tabulates  the  costs  associated 
with  training  and  evaluates  the  training  capacity  of  the  training  system  (i.e.,  the  ability  of  the  training  system  to 
provide  training  to  a  given  number  of  airmen). 

Mitchell,  J.  L.,  Vaughan,  D.  S.,  Knight,  J.  R.,  Rueter,  F.  H.,  Fast,  J.,  Haynes  W.  R.,  &  Bennett,  W.  R. 

(1992,  June).  Training  decisions  technology  analysis.  (AL-TP-1 992-0026).  Brooks  AFB,  TX: 
Technical  Training  Research  Division,  Armstrong  Laboratory,  Human  Resources  Directorate. 

The  Training  Decisions  System  (TDS)  is  a  computer-based  decision  support  technology  which  has  been 
developed  to  provide  a  more  integrated  approach  to  Air  Force  training  planning  and  development.  This 
technology  was  developed  in  an  earher  research  and  development  (R&D)  effort  which  resulted  in  a  proof-of- 
concept  system.  The  present  project  examines  the  technologies  and  scientific  innovations  involved  in  the  TDS 
both  from  the  standpoint  of  separate  subsystem  technologies  and  from  the  viewpoint  of  an  integrated  decision 
support  system.  The  procedures  and  software  from  the  proofrof-concept  TDS  were  exercised  to  identify  problems 
in  programs  and  software  documentation;  the  problems  recognized  in  this  exercise  of  the  system  were  corrected 
and  new  programs  and  data  files  created.  Extensive  sensitivity  analyses  were  conducted  to  estabhsh  which  types 
of  variables  had  the  greatest  impact  on  output  products  (total  AFS  training  costs  and  estimates  of  on-the-job 
training  edacity  of  representative  field  units).  Additicaial  validation  work  was  planned  as  was  a  study  to  examine 
the  systematic  biases  in  TDS  data  bases.  The  scientific  contributions  of  the  TDS  R&D  project  are  examined  and 
some  conclusions  drawn  as  to  how  TDS  relates  to  state-of-the-art  technologies  in  several  academic  and  technical 
areas.  Finally,  several  potential  ^plications  of  the  TDS  technolo©^  are  recommended  in  terms  of  future  training 
management  research  as  well  as  direct  ^pheation  other  mihtary  services  and  the  civihan  sector. 

Coccia,  T.,  Mitchell,  J.  L.,  Knight,  J.  R.,  &  Shrum,  R.  C.  (1992,  October).  Development  of  a  career  field 
training  management  plan  for  the  electronic  computer  and  switching  systems  specialty.  Proceedings 
of  the  34th  Annual  Conference  of  the  Military  Testing  Association  (pp.  831-836).  San  Diego,  CA: 
Navy  Personnel  Research  and  Development  Center. 

A  joint  study  of  the  Electronic  Computer  and  Switching  Systems  specialty  was  undertaken  by  the  USAF 
Occupational  Measurement  Squadron  and  the  Techmeal  Training  Research  Division  of  the  Armstrong 
Laboratory,  as  reported  at  the  1991  Military  Testing  Association  Conference.  The  parallel  occupational  analysis 
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and  Training  Decision  System  (TDS)  studies  were  both  aimed  at  assisting  the  AFS  305X4  Functional  Manager 
in  making  critical  decisions.  The  TDS  results  were  used  to  model  the  current  state  of  the  specialty  as  well  as 
several  alternative  structures  proposed  by  an  Subject-Matter  Expert  panel  which  met  in  May  1992.  Data  from 
both  efforts  will  be  briefed  to  a  Utilization  and  Training  Workshop  (U&TW)  for  the  specialty  in  late  1992,  and 
the  informaticMi  will  be  used  by  U&TW  participants  to  make  decisions  about  the  fiiture  course  of  training  in  the 
specialty.  Some  of  the  data  from  the  occupational  analysis  and  the  TDS  study  were  also  used  to  draft  sections 
of  a  Career  Field  Management  Plan  (CFMP),  a  newly-required  planning  document  which  highlights  expected 
changes  in  manpower,  personnel,  and  training  programs  over  the  next  five  years.  A  CFMP  will  be  required  of 
all  Air  Force  specialties  in  the  near  fiiture;  the  present  study  has  demonstrated  that  both  occupational  analysis 
and  TDS  data  can  be  useful  in  CFMP  development. 

Mitchell,  J.  L.,  Phalen,  W.  J.,  &  Hand,  D.  K.  (1992,  October).  Multilevel  occupational  analysis:  hierarchies 
of  tasks,  modules,  jobs,  and  specialties.  In  the  symposium.  Organizational  analysis  issues  in  the 
military  (Ruck,  H.  W.,  chair).  Proceedings  of  the  34th  Annual  Conference  of  the  Military  Testing 
Association  (pp.  729-734).  San  Diego,  CA:  Navy  Personnel  Research  and  Development  Center. 

There  are  extensive  amounts  of  task-level  information  now  available  on  most  military  occupations  which 
are  used  effectively  for  a  variety  of  purposes.  While  detailed  task-level  information  is  critical  for  certain  uses, 
such  as  training  development  cr  selecting  topics  for  promotion  tests,  such  data  may  be  too  complex  and  specific 
for  other  purposes;  for  example,  for  faciUtating  management  macro-level  decision  making  or  evaluation  of 
possible  impacts  of  organizational  restmeturing.  Organizing  task  information  into  task  modules,  jobs,  and  higher 
order  categories  allows  the  data  to  be  applied  to  more  global  issues  and  problems  and  can  be  used  to  develop 
realistic  models  or  simulations  of  occupational  structures  and  requirements.  Existing  data  already  permit 
comprehensive  organizational  modeUng;  some  present  analyses  involve  multiple  specialties,  multiple  categories 
ofperscmnel  (enlisted,  officer,  civilian),  or  even  multiple  services  (interseiwice  or  joint  service  projects).  Given 
the  ccMisiderable  value  of  task-based  information  and  analyses,  multi-level  studies  focused  on  task  modules  and 
other  higher  order  groupings  have  considerable  potential  for  applications  in  modeUng  mihtary  organizations  to 
assist  military  decision  makers  in  evaluating  proposed  organizational  restructuring,  interventions,  and/or  other 
organizational  changes.  This  paper  \vill  highlight  new  ASCII  Comprehensive  Occupational  Data  Analysis 
Programs  (CODAP)  technology  which  permits  analysis  of  occupational  data  at  a  number  of  different  levels  of 
specificity  or  from  a  variety  of  viewpoints.  The  new  programs  permit  grouping  of  task-level  data  into  meaningful 
clusters  either  statistically  or  on  the  basis  of  some  rational  organizational  principle.  Implications  of  multi-level 
data  for  simulating  organizational  change  will  be  discussed. 

Vaughan,  D.  S.,  Yadrick,  R.  M.  (1992,  October).  An  organizational  analysis  simulation  technology.  In  the 
symposium.  Organizational  analysis  issues  in  the  military  (Ruck,  H.  W.,  chair).  Proceedings  of  the 
34th  Annual  Conference  of  the  Military  Testing  Association  (pp.  735-740).  San  Diego,  CA;  Navy 
Personnel  Research  and  Development  Center. 

One  ^proach  to  addressing  the  quantifiable  reduction  in  effect  size  which  is  attributable  to  the  moderating 
effect  of  linkages  is  to  develop  a  model  which  quantifies  the  contribution  of  changes  in  performance  and/or 
productivity  at  one  level  and  at  other  proximal  level.  Such  a  model  would  express  important  macro-level  outcome 
variables  (e.g.,  organizational  productivity,  mission  effectiveness)  as  functions  of  important  causal  variables. 
These  include  both  micro-level  variables  that  are  directly  manipulated  in  organizational  interventions  and  more 
macro-level  variables,  such  as  business  and  economic  conditions.  This  type  of  model  would  be  very  useful  for 
exploring  the  quantitative  relationships  among  events  at  various  levels  of  abstraction.  Such  a  model  could  be 
used  to  determine  the  maximum  impact  that  micro-level  organizational  interventions  can  reasonably  be  expected 
to  have  on  macro-level  outcome  variables,  relative  to  other  uncontrolled  events.  This  paper  describes  the 
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Training  Decisions  System  (TDS),  a  simulation  technology  which  relates  events  at  various  levels  of  abstraction. 
The  model  relates  miCTO-level  personnel  events  to  macro-level  outcome  variables.  Data  for  the  TDS  comes  from 
a  variety  of  sources,  induing  job  analysis,  existing  manpower,  personnel,  and  training  data  bases,  and 
subject-matter  experts' judgments.  The  TDS  estimates  overall  job  flows,  training  requirements,  and  training  costs 
from  individual  job,  task,  and  training  assignments,  based  on  job  analysis  data.  The  TDS  model  first  simulates 
the  flow  of  individuals  through  jobs,  with  task  performance  requirements,  and  formal  training  courses.  From 
these  individual  events,  the  model  estimates  task-level,  on-the-job  training  events.  Finally,  the  model  estimates 
overall  training  resource  requirements,  costs,  and  capacities  from  the  task-level  events. 

Ward,  J.  H.,  Jr.,  Vaughan,  D.  S.,  MtcheU,  J.  L.,  DriskiU,  W.  E.,  &  Ruck,  H.  W.  (1992,  October).  The 
ultimate  person-job  match:  a  k^  to  fixture  worker  productivity.  Proceedings  of  the  34th  Annual 
Conference  of  the  Military  Testing  Association  (pp.  849-854).  San  Diego,  CA:  Navy  Personnel 
Research  and  Development  Center. 

Given  current  trends  of  decreasing  budgets,  reduced  manpower,  and  escalating  personnel  benefits  and 
training  costs,  it  is  imperative  that  the  mihtary  services  consider  human  resource  poUcy  alternatives  that  will 
enhance  individual  productivity  and  optimize  personal  growth  and  development.  In  the  long  run,  a  highly 
effective  approach  would  be  to  focus  on  improving  the  match  between  job  incumbents  skills  and  job 
requirements.  This  approach  begins  with  the  enhancement  of  the  kinds  of  information  available  about  the  jobs 
and  the  people  as  they  enter  occupations.  New  technologies  are  under  development  to  improve  the  definition 
of  what  skills,  knowledge,  and  abilities  are  required  for  each  occupation;  such  technology  could  easily  be 
extended  to  be  job-  or  position-specific.  Using  tWs  new  technology,  a  better  job  requirements  data  base  could 
be  develqjed  as  the  Base  Training  System  (BTS,  formerly  the  Advanced  On-the-job  Training  System  or  AOTS) 
is  implemented.  Advanced  utilization  and  training  models,  such  as  those  developed  in  the  Training  Decisions 
Support  Technolo©^  line  of  research,  could  be  integrated  as  well,  to  help  functional  managers  anticipate  future 
changes  in  jobs  and  plan  for  possible  career  field  transitions.  Given  recent  developments  in  computers  and 
modeling  systems,  the  merging  of  all  these  technologies  into  an  integrated  human  resources  management  system 
becomes  both  possible  and  practical.  The  ultimate  system  should  be  able  to  optimize  on  multiple  functions  so 
as  to  produce  improvements  in  a  variety  of  outcome  variables,  such  as  individual  productivity,  organizational 
goal  achievement,  personal  growth  and  development,  and  classification  structure  stabihty. 

Mitchell,  J.  L.,  Yadrick,  R.  M.,  &  Bennett,  W.  R.  (1993).  Estimating  training  requirements  from  job  and 
training  models.  Military  Psychology  5(l):l-20. 

New  methods  were  developed  for  estimating  the  annual  training  requirements  of  individual  occupations 
(Air  Force  specialties).  These  methods  facihtate  the  generation  of  quantitative  estimates  of  formal  classroom, 
hands-cm,  self-study,  and  <Mi-the-job  training  needed  as  a  basis  for  calculating  annual  training  requirements  and 
for  assessing  the  capacity  of  representative  field  units  to  provide  such  training.  Using  a  simulation  approach  to 
estimate  training  requirements  offers  considerable  flexibility  for  assisting  decision  makers  in  evaluating  the 
possible  impact  of  changes  on  current  job  structures  and  training  programs. 

Agee,  R.,  Martino,  K.  B.  W.,  Lamb,  T.  A.,  &  Knight,  J.  R.  (1993, 15  June).  An  operational  test  of  OA  and 
TIDES  data  for  MPT  decision  making.  In  the  symposium  (Ruck,  H.  W.,  Chair)  Military  Occupational 
Analysis:  Issues  &  Advances  in  Research  and  Apphcation.  Proceedings  of  the  Eighth  International 
Occupational  Analysis  Workshop  (pp.  27-3 1 ).  San  Antonio,  TX:  USAF  Occupational  Measurement 
Squadron. 
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This  presentation  highlights  the  synergistic  effect  of  combining  occupational  analysis  (OA)  technology 
and  Training  Impact  Decision  System  (TIDES)  models  and  methodologies.  The  apphcation  of  OA  and  TIDES 
products  to  manpower,  personnel  and  training  (MPT)  decision  making  in  the  recent  AFS  305X4,  Electronic 
Computer  &  Switching  Systems,  Utilization  and  Training  Workshop  (U&TW)  is  discussed  as  are  specific 
contributions  of  collaborative  efforts  among  the  USAF  Occupational  Measurement  Squadron,  Armstrong 
Laboratory,  and  the  R&D  contractor  team.  Systematic  feedback  data  (ratings  and  comments)  fi'om  U&TW 
participants  were  gathered  to  evaluate  the  operational  use  of  OA  and  TIDES  information. 

Bennett,  W.  R.,  Jr.,  Vaughan,  D.  S.,  &  Ruck,  H.  W.  (1993,  June).  Military  occupational  analysis:  Future 
directions.  In  the  symposium.  Military  occupational  analysis;  Issues  and  advances  in  research  and 
application,  H.  W.  Ruck,  Chair.  Proceedings  of  the  Eighth  International  Occupational  Analyst 
Workshop  (pp.  32-36).  San  Antonio,  TX:  USAF  Occupational  Measurement  Squadron. 

This  paper  highhghts  several  potential  future  directions  in  occupational  analysis.  Specifically, 
improvements  to  current  occupational  analysis  practices  are  proposed  and  discussed.  In  addition,  new  uses  of 
occupational  analysis  methods  for  such  things  as  aggregating  task-level  information,  d5mamic  modeling  of 
organizational  change,  new  training  needs  assessment  activities,  and  deriving  estimates  of  the  cost  benefit  of 
interventions  at  an  organizational  level  are  discussed.  Finally,  several  key  issues  related  to  advancing  the 
research  and  development  of  occupational  analysis  methods  and  applications  are  highhghted. 

Vaughan,  D.  S.,  Grimes,  G.  R.,  &  Knight,  J.  R.  (1993,  November).  Evaluating  the  impact  of  policies  on 
organizations;  Increasing  assignment  length.  In  the  symposium.  Organizational  Analyses  and 
Research:  Policy  Issues,  Modeling,  and  Future  Technologies,  W.  R.  Beimett,  Jr.,  Chair.  Proceedings 
of  the  35th  Annual  Conference  of  the  Military  Testing  Association  (pp.  37-42).  Williamsburg,  VA; 
US.  Coast  Guard  Headquarters,  Occupation^  Analysis  Program. 

This  presentaticMi  highlighted  the  synergistic  effect  of  using  occupational  analysis  (OA)  data  to  inform  an 
organizational  simulation  technology  for  use  in  evaluating  policy  impacts  on  orgardzations.  The  utility  of 
integrating  OA  data  and  organizational  simulations  for  manpower,  persoimel,  and  training  (MPT)  decision 
making  were  highlighted  using  an  actual  policy  issue,  increasing  assignment  length,  and  simulation  outcomes 
fi'om  recent  Electronic  Computer  &  Switching  Systems  and  Aerospace  Propulsion  utilization  and  training 
workshop  (U&TW)  projects.  The  implications  for  an  integrated  approach  to  policy  analysis  were  explored  and 
evaluated. 

Vaughan,  D.  S.  &  Beimett,  W.  R.,  Jr.  (1 993,  November).  Toward  a  unified  theory  of  work:  Organizational 
simulations  and  poUcy  analyses.  In  the  symposium.  Organizational  Analyses  and  Research:  Policy 
Issues,  Modeling,  and  Future  Technologies,  W.  R.  Bennett,  Jr.,  Chair.  Proceedings  of  the  35th 
Annual  Conference  of  the  Military  Testing  Association  (pp.  31-36).  Wilhamsburg,  VA:  US.  Coast 
Guard  Headquarters,  Occupational  Analysis  Program. 

The  miUtary  environment  is  constantly  changing  and  restructuring.  MPT  planning  and  management  is 
increasingly  crucial  to  maintaining  the  mission  readiness  of  the  forces.  The  DoD  needs  an  integrated  MPT 
planning  and  management  system.  We  beheve  that  a  unified  theory  of  work  is  needed  to  provide  a  Jframework 
and  to  guide  and  focus  related  R&D.  This  uniBed  theory  of  woik  will  connect  theories  of  human  traits  and  states, 
theories  of  task  and  job  characteristics,  theories  of  job/task  performance,  and  perhaps  theories  of  organizational 
behavior.  For  example,  Mitchell  &  Driskill  (1986)  have  proposed  a  theory  that  relates  training  to  task 
performance,  via  a  series  of  intervening  and  exogenous  variables.  Such  a  theory  could  be  extended  to  encompass 
individual  differences  among  workers  and  tasks,  as  well  as  key  organizational  and  environmental  variables.  This 
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presentation  will  explore  key  issues  associated  with  a  unified  theory  approach  to  MPT  modeling  and  decisiori 
making.  Further,  basic  research  and  development  needs  required  for  such  an  integrated  approach  will  be 
highlighted  and  discussed. 

Ward,  J.  H.,  Jr.,  Mitchell,  J.  L.,  Weissmuller,  J.  J.,  &  Vaughan,  D.  S.  (1993,  November).  A  software 

system  for  modehng  jobs  and  people  characteristics  to  help  optimize  individual  and  organizational 
objectives.  In  the  symposium.  Organizational  Analyses  and  Research:  Policy  Issues,  Modeling, 
and  Future  Technologies,  W.  R.  Bennett,  Jr.,  Chair.  Proceedings  of  the  35th  Annual  Conference  of 
the  Military  Testing  Association  (pp.  25-30).  Williamsburg,  VA:  U.S.  Coast  Guard  Headquarters, 
Occupational  Analysis  Program. 

There  is  a  critical  need  for  the  development  of  a  general  utility  software  package  which  would  permit  both 
policy  c^turing  and  policy  specifying  analyses  through  modeling  of  individual  people  characteristics  and  job 
properties.  Such  software  should  be  functional  at  multiple  levels  of  abstraction;  at  an  operational  level,  for 
person-job  matching,  and  at  an  orgamzational  planning  level  for  executive  assessment  of  alternative  strategies 
for  mission  accomplishment  or  ad^tation  to  changes.  Planners  and  decision-makers  need  to  be  able  to  evaluate 
optimal  ways  of  allocating  available  people  to  existing  or  anticipated  work,  and  individuals  need  to  order  their 
preferences  based  on  available  or  expected  work  and  possible  training.  The  needed  software  system  would 
extend  traditional  person-job  match  technology  to  become  a  multi-level  simulation  and  analysis  system  which 
could  be  potentially  very  usefiil  in  fiizzy  budgeting  or  programming,  and  in  planning  for  military  to  civilian 
conversions  (individuals,  units,  bases,  etc.).  We  strongly  advocate  an  expeditious  research  &  development  effort 
for  the  design  and  tryout  of  such  a  software  system. 

Stone,  B.,  Weissmuller,  J.J.,  &  Mitchell,  J.L.  (1995,  May).  Occupational  analysis  data  and  MPT  -  related 
research  and  development.  In  the  symposium.  Military  occupational  analysis:  Research  and 
application  for  manpower,  personnel,  and  training,  A.M.  Smith,  n  &  W.  Beimett,  Jr.,  Chairs. 
Proceedings  of  the  Ninth  International  Occupational  Analyst  Workshop  (pp.  1 9-26).  San  Antonio, 
TX:  Air  Force  Occupational  Measurement  Squadron. 

A  number  of  research  and  development  (R&D)  contracts  are  currently  underway  which  make  use  of 
occupational  survey  data  for  a  wide  variety  of  purposes  at  vaiying  levels  of  specificify  and  organizational 
structure.  Since  all  of  these  projects  use  task-based  occupational  analysis  as  a  starting  point,  most  of  the 
technologies  are  highly  compatible  and  can  work  synergistically;  they  could  easily  be  integrated  into  an  overall 
militaiy  MPT  support  system  to  deal  with  a  wide  range  of  decision  making,  manpower  modeling,  personnel 
classification  specification,  training  development  and  evaluation,  MPT  systems  operation,  research  and 
development,  and  organizational  productivity  enhancement  activities.  Some  of  these  evolving  technologies,  such 
as  RISC-bask  CODAP,  computer-based  surveying,  improved  job  and  task  clustering,  training  impact  decision 
support  (i.e.,  TIDES),  and  organizational  modeling  hold  substantial  promise  to  greatly  enhance  the  value  and 
usefulness  of  mihtaiy  occupational  analysis  programs. 


Bennett,  T.,  Zukor,  K.,  Teachout,  M.S.,  Smith,  AM.  H,  &  Bennett,  W.,  Jr.  (1995,  May).  Occupational 

analysis  data  and  training-related  research  and  development.  In  the  symposium.  Military  occupational 
analysis:  Research  and  application  for  manpower,  personnel,  and  training,  A  M.  Smith,  n  &  W. 
Bennett,  Jr.,  Chairs.  Proceedings  of  the  Ninth  International  Occupational  Analyst  Workshop  (pp.64- 
68).  San  Antonio,  TX:  Air  Force  Occupational  Measurement  Squadron. 

Current  education  and  training  research  focuses  on  macro  issues  related  to  modeling  and  planmng  jobs 
and  training  flow  for  entire  career  fields,  and  more  micro  issues  related  to  the  knowledge,  skills,  abilities,  and 
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tasks  associated  with  training  and  job  performance.  Current  research  and  development  which  addresses  a  number 
of  more  macro  issues  in  training  and  education  needs  assessment  and  evaluation  is  the  Training  Impact  Decision 
System  (TIDES).  Similarly,  the  Training  Efficiency  and  Effectiveness  Methodology  (TEEM)  is  designed  to 
address  more  micro  issues.  Recent  activities  from  exh  of  these  major  efforts  were  discussed  in  terms  of  how  they 
capture  and  portr^  occupational  analysis  data  for  use  in  career  field  decision  making  and  for  course  development 
and  revision. 

Agee,  R.,  Martino,  K.B.W.,  Lamb,  T.,  &  Kni^t,  J.R.  (1 995,  August).  An  operational  test  of  OA  and  TIDES 
data  for  MPT  decision-making.  In  Military  occupational  analysis:  Issues  and  advances  in  research  and 
application,  W.  Bennett,  Jr.,  &  J.  L.  Mtchell,  Editors  (AL/HR-TR-1995-0131;  pp.  34-44).  Brooks 
AFB,  TX:  Technical  Training  Research  Division,  Armstrong  Laboratory  Human  Resources 
Directorate. 

The  Armstrong  Laboratory/Human  Resources  Directorate  (AL/HR)  conducted  an  operational  test  of  the 
Training  hnpxt  Decision  System  (TIDES;  previously  the  Training  Decisions  System)  in  conjunction  with  a 
cotitracting  team  led  by  McDonnell  Douglas  Aerospxe.  Occupational  analysis  (OA)  expertise  and  operational 
suppcnt  were  provided  by  the  Air  Force  Occupational  Measurement  Squadron  (AFOMS).  This  operational  test 
was  highly  successfiil;  TIDES  can  be  used  to  dynamically  model  the  long-term  personnel  and  training  issue 
impacts  on  an  AFS  for  various  combinations  of  training  content,  sites,  and  proposed  policy  changes  within  the 
training  system.  TIDES  is  a  powerful  methodology  to  aid  key  decision-makers  at  the  MAJCOM  and  USAF 
levels  in  determining  the  optimal  utihzation  of  resources  necessary  to  support  the  mission  of  an  AFS. 

Bennett,  W.,  Jr.,  Vaughan,  D.S.,  &  Ruck,  H.W.  (1995,  August).  Military  occupational  analysis:  Future 
directions.  In  Military  occupational  analysis:  Issues  and  advances  in  research  and  apphcation,  W. 
Bennett,  Jr.,  &  J.  L.  MitcheU,  Editors  (AL/HR-TR-1995-0131;  pp.  45-55).  Brooks  AFB,  TX: 

Technical  Training  Research  Division,  Armstrong  Laboratory  Human  Resources  Directorate. 

Typically,  job  analysis  data  are  used  to  portr^  the  current  state  of  tasks  within  jobs  in  an  organization  for 
the  analysis  of  personnel  recruitment,  selection,  placement,  training  and  development,  and  performance  appraisal. 
There  are  key  areas  of  opportunity  for  future  research  in  occupational  analysis  and  apphcation.  These  include 
improvements  to  existing  OA  prxtice  and  extending  current  occupational  modehng  for  multilevel  analysis  and 
dynamic  organizational  modehng.  A  unified  theory  of  work  is  needed  to  provide  a  framework  for  future  R&D 
and  to  guide  and  focus  future  OA  apphcations. 

Mitchell,  J.L.,  Knight,  J.R.,  Luster,  C.,  Stone,  B.,  Turner,  K.,  Hand,  D.,  Smith,  AM.,  H,  Bennett,  T. 

Press).  Imphcations  of  using  mixed  civihan-mihtary  samples  in  developing  career  field  education  and 
training  plans.  In  the  symposium.  Issues  and  advances  in  task-based  occupational  research  and 
development  for  manpower,  personnel,  and  training,  R.  B.  Gould,  Chair.  Proceedings  of  the  37th 
Annual  Conference  of  the  International  Military  Testing  Association.  Toronto,  Ontario,  Canada: 
Canadian  Forces  Personnel  Apphed  Research  Unit,  October  1995. 

An  occupational  analysis  of  the  Pavements  Maintenance  and  Construction  Equipment  specialty  included 
both  Air  Force  mihtary  and  civihan  personnel;  this  study  provided  the  opportunity  to  conduct,  for  the  first  time, 
a  joint  study  utilizing  the  Training  Impxt  Decision  System  (TIDES)  with  civihan  employees.  The  impact  of 
including  civihans  in  the  study  on  task  and  job  clusters  was  assessed,  as  well  as  broader  issues  of  TIDES  data 
collection  or  estimation.  The  five  civihan  series  involves  were  differently  represented  in  the  sample  which 
comphcates  modeling  of  job  and  training  probabihties,  as  is  required  for  the  estimation  of  total  annual  costs  for 
training  The  stucfy  is  ou-going,  with  final  results  due  early  in  1 996  when  a  Utihzation  and  Training  Workshop 
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for  this  specialty  will  be  convened  to  develop  a  career  field  education  and  training  plan  (CFETP)  using  the 
CFETP  template  available  in  TIDES. 

Stone,  B,  Turner,  K.,  &  Curry,  G.  (In  Press).  Determining  the  representativeness  of  outcomes  for  an  entity- 
based  training  simulation  model.  In  the  symposium.  Using  occupational  information  for 
organizational  analysis:  Issues  and  recent  apphcations,  R.  M.  Yadrick,  Chair.  Proceedings  of  the 
37th  Annual  Conference  of  the  International  Military  Testing  Association.  Toronto,  Ontario, 

Canada:  Canadian  Forces  Personnel  AppUed  Research  Unit,  October  1995. 

A  recent  research  effort  focused  on  developing  a  methodology  to  determine  the  optimum  time  period  for 
stopping  a  time-series  entity  based  simulation;  this  work  was  conducted  using  the  Training  Impact  Decision 
System  (TIDES)  which  is  an  entity-based  job  and  training  simulation  model.  The  primary  objective  was  to 
obtain  and  insure  representative  outcomes  fi'om  the  simulation  for  baseline  and  alternative  scenario  simulations. 
Mathematical  equations  for  the  identification  of  the  optimum  time  period  for  stopping  the  simulation  were 
developed  and  imbedded  into  the  TIDES  modehng  system.  These  equations  were  also  used  to  determine  the 
number  of  repetitions  required  to  insure  representativeness  of  transition  periods  for  policy  changes  implemented 
in  alternative  applications.  Confidence  intervals  were  developed  for  outcome  values  of  the  simulation  for 
comparison  of  steady  state  and  transition  period  outcome  values. 

Bennett,  W.,  Jr.,  Perrin,  B.,  &  Yadrick,  R.M.  (In  Press).  Using  organizational  simulations  to  estimate  the 
utiUty  of  interventions  and  change.  In  the  symposium.  Using  occupational  information  for 
organizational  analysis:  Issues  and  recent  applications,  R.  M.  Yadrick,  Chair.  Proceedings  of  the 
37th  Annual  Conference  of  the  International  Military  Testing  Association.  Toronto,  Ontario, 

Canada:  Canadian  Forces  Personnel  Applied  Research  Unit,  October  1995. 

Recently  there  has  been  considerable  interest  in  the  use  of  computer  simulations  to  examine  the  impact 
of  organizational  interventions  and  to  quantify  the  effects  of  moderators  by  making  relationships  between 
these  interventions  and  multilevel  outcomes  explicit.  The  potential  uses  of  an  organizational  simulation  were 
examined  for  (a)  minimizing  requirements  for  additional  training  as  a  result  of  new  personnel  assignments; 
and  (b)  estimating  the  requirements  for  additional  training,  providing  a  measure  of  training  utility,  expressed 
as  changes  in  costs,  changes  in  proficiency,  and  the  impact  of  training  requirements  on  available  training 
resources.  In  addition,  recent  results  fi'om  the  use  of  the  simulation  to  assess  the  utility  of  chapge  fiom  job 
restructuring/reorganization  and  changes  to  training  course  content  and  delivery  methods  were  also 
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